Balancing Treatment and Control Groups in Quasi-Experiments: An Introduction to Propensity Scoring

Authors


  • We thank Howard J. Klein, Raymond A. Noe, Chongwei Wang, Donald E. Powers, and Donald A. Rock for generously sharing their data with us for illustrating propensity scoring analyses.

Correspondence and requests for reprints should be addressed to Brian S. Connelly, Department of Management, 1265 Military Trail, University of Toronto, Toronto, ON M1C 1A4, Canada; brian.connelly@utoronto.ca.

Abstract

Organizational and applied sciences have long struggled with improving causal inference in quasi-experiments. We introduce organizational researchers to propensity scoring, a statistical technique that has become popular in other applied sciences as a means for improving internal validity. Propensity scoring statistically models how individuals in a quasi-experiment have been assigned to conditions in order to estimate treatment effects among individuals with approximately equal probabilities of receiving the treatment. If propensity scores are created from relevant covariates, matching on the propensity score makes treatment assignment ignorable and approximates a true experimental design. We illustrate how matching on the propensity score can be applied by using 2 examples: examining the effects of online instruction and estimating the benefits of preparatory coaching for the SAT. In both cases, propensity-scoring methods effectively reduced inequivalence between treatment and control groups on many variables. Propensity scoring stands out as a valuable technique capable of improving causal inference from many of organizational research's quasi-experiments.

Ancillary