Get access

Propensity score balance measures in pharmacoepidemiology: a simulation study

Authors

  • M. Sanni Ali,

    1. Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, University of Utrecht, Utrecht, The Netherlands
    Search for more papers by this author
  • Rolf H. H. Groenwold,

    1. Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, University of Utrecht, Utrecht, The Netherlands
    2. Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
    Search for more papers by this author
  • Wiebe R. Pestman,

    1. Departamento de Matematica, Universidade Federal de Santa Catarina, Florianopolis, Santa Catarina, Brazil
    Search for more papers by this author
  • Svetlana V. Belitser,

    1. Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, University of Utrecht, Utrecht, The Netherlands
    Search for more papers by this author
  • Kit C. B. Roes,

    1. Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
    Search for more papers by this author
  • Arno W. Hoes,

    1. Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
    Search for more papers by this author
  • Anthonius de Boer,

    1. Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, University of Utrecht, Utrecht, The Netherlands
    Search for more papers by this author
  • Olaf H. Klungel

    Corresponding author
    1. Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, University of Utrecht, Utrecht, The Netherlands
    2. Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
    • Correspondence to: O. H. Klungel, Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences, University of Utrecht, PO Box 80082, 3508 TB Utrecht, The Netherlands. Email: O.H.Klungel@uu.nl

    Search for more papers by this author

  • The abstract has been presented in the 28th ICPE, 23–26 August 2012, Barcelona, and published in Pharmacoepidemiology and Drug Safety, 2012; 21: (Suppl. 3): 36–37 DOI: 10.1002 as “Evaluating propensity score balance measures in typical pharmacoepidemiologic settings”.

ABSTRACT

Background

Conditional on the propensity score (PS), treated and untreated subjects have similar distribution of observed baseline characteristics when the PS model is appropriately specified. The performance of several PS balance measures in assessing the balance of covariates achieved by a specific PS model and selecting the optimal PS model was evaluated in simulation studies. However, these studies involved only normally distributed covariates. Comparisons in binary or mixed covariate distributions with rare outcomes, typical of pharmacoepidemiologic settings, are scarce.

Methods

Monte Carlo simulations were performed to examine the performance of different balance measures in terms of selecting an optimal PS model, thus reduction in bias. The balance of covariates between treatment groups was assessed using the absolute standardized difference, the Kolmogorov–Smirnov distance, the Lévy distance, and the overlapping coefficient. Spearman's correlation coefficient (r) between each of these balance measures and bias were calculated.

Results

In large sample sizes (n ≥ 1000), all balance measures were similarly correlated with bias (r ranging between 0.50 and 0.68) irrespective of the treatment effect's strength and frequency of the outcome. In smaller sample sizes with mixed binary and continuous covariate distributions, these correlations were low for all balance measures (r ranging between 0.11 and 0.43), except for the absolute standardized difference (r = 0.51).

Conclusions

The absolute standardized difference, which is an easy-to-calculate balance measure, displayed consistently better performance across different simulation scenarios. Therefore, it should be the balance measure of choice for measuring and reporting the amount of balance reached, and for selecting the final PS model. Copyright © 2014 John Wiley & Sons, Ltd.

Ancillary