Chapter

11 Research Designs for Program Evaluation

Research Methods in Psychology

II. RESEARCH METHODS IN SPECIFIC CONTENT AREAS

  1. Vivian C. Wong PhD1,
  2. Coady Wing PhD2,
  3. Peter M. Steiner PhD3,
  4. Manyee Wong PhD4,
  5. Thomas D. Cook PhD5

Published Online: 26 SEP 2012

DOI: 10.1002/9781118133880.hop202011

Handbook of Psychology, Second Edition

Handbook of Psychology, Second Edition

How to Cite

Wong, V. C., Wing, C., Steiner, P. M., Wong, M. and Cook, T. D. 2012. Research Designs for Program Evaluation. Handbook of Psychology, Second Edition. 2:II:11.

Author Information

  1. 1

    Northwestern University, Institute for Policy Research, Evanston, Illinois, USA

  2. 2

    University of Illinois at Chicago, School of Public Health, Chicago, Illinois, USA

  3. 3

    University of Wisconsin, Department of Educational Psychology, Madison, Wisconsin, USA

  4. 4

    American Institutes for Research, Chicago, Illinois, USA

  5. 5

    Institute for Policy Research, Evanston, Illinois, USA

Publication History

  1. Published Online: 26 SEP 2012

Abstract

Over the past two decades, the program evaluation literature has made great advances on improving methodological approaches for establishing causal inference. The two most significant developments include establishing the primacy of design over statistical adjustment procedures for making causal inferences, and using potential outcomes to specify the exact causal estimands produced by the research designs. This chapter presents four research designs for assessing program effects-the randomized experiment, the regression-discontinuity, the interrupted time series, and the nonequivalent comparison group designs. For each design, we examine basic features of the approach, use potential outcomes to define causal estimands produced by the design, and highlight common issues to consider when using the design in the field. Whenever possible, we use examples to illustrate how these designs have been used to assess program effects. We conclude by suggesting broader issues in program evaluation that the next generation of evaluators should consider.

Keywords:

  • program evaluation;
  • experiments;
  • regression-discontinuity;
  • interrupted time series;
  • propensity score matching