Volume 3, Issue 3
Original Article

A standardized mean difference effect size for single case designs

Larry V. Hedges

Northwestern University, Evanston, IL, USA

Search for more papers by this author
James E. Pustejovsky

Corresponding Author

Northwestern University, Evanston, IL, USA

James E. Pustejovsky, Northwestern University.

E‐mail: pusto@u.northwestern.edu

Search for more papers by this author
William R. Shadish

University of California, Merced, CA, USA

Search for more papers by this author
First published: 14 August 2012
Citations: 127

Abstract

Single case designs are a set of research methods for evaluating treatment effects by assigning different treatments to the same individual and measuring outcomes over time and are used across fields such as behavior analysis, clinical psychology, special education, and medicine. Emerging standards for single case designs have focused attention on the need for effect sizes for summarizing and meta‐analyzing findings from the designs; although many effect size measures have been proposed, there is little consensus regarding their use. This article proposes a new effect size measure for single case research that is directly comparable with the standardized mean difference (Cohen's d) often used in between‐subjects designs. Techniques are provided for estimating the new effect size, as well as its variance, from balanced or unbalanced treatment reversal designs. The proposed estimation methods are further evaluated using a simulation study and then demonstrated in two applications. Copyright © 2012 John Wiley & Sons, Ltd.

Number of times cited according to CrossRef: 127

  • Developing and training mental toughness in sport: a systematic review and meta-analysis of observational studies and pre-test and post-test experiments, BMJ Open Sport & Exercise Medicine, 10.1136/bmjsem-2020-000747, 6, 1, (e000747), (2020).
  • A primer for using multilevel models to meta‐analyze single case design data with AB phases, Journal of Applied Behavior Analysis, 10.1002/jaba.698, 53, 3, (1799-1821), (2020).
  • Multitiered Systems of Support in Preschool Settings: A Review and Meta-Analysis of Single-Case Research, Topics in Early Childhood Special Education, 10.1177/0271121419899720, (027112141989972), (2020).
  • From Boulder to Stockholm in 70 Years: Single Case Experimental Designs in Clinical Research, The Psychological Record, 10.1007/s40732-020-00402-5, (2020).
  • Treatment Burst Data Points and Single Case Design Studies: A Bayesian N-of-1 Analysis for Estimating Treatment Effect Size, Perspectives on Behavior Science, 10.1007/s40614-020-00258-8, (2020).
  • Assessing consistency of effects when applying multilevel models to single-case data, Behavior Research Methods, 10.3758/s13428-020-01417-0, (2020).
  • The nociceptive flexion reflex: a scoping review and proposed standardized methodology for acquisition in those affected by chronic pain, British Journal of Pain, 10.1177/2049463720913289, (204946372091328), (2020).
  • Efficacy of a Virtual Reality Biofeedback Game (DEEP) to Reduce Anxiety and Disruptive Classroom Behavior: Single-Case Study, JMIR Mental Health, 10.2196/16066, 7, 3, (e16066), (2020).
  • Estimating effect size with respect to variance in baseline to treatment phases of single-case experimental designs: A Bayesian simulation study, Evidence-Based Communication Assessment and Intervention, 10.1080/17489539.2020.1738625, (1-13), (2020).
  • A Multiple-Baseline Evaluation of Acceptance and Commitment Therapy Focused on Repetitive Negative Thinking for Comorbid Generalized Anxiety Disorder and Depression, Frontiers in Psychology, 10.3389/fpsyg.2020.00356, 11, (2020).
  • Indexing effects in single-case experimental designs by estimating the percent of goal obtained, Evidence-Based Communication Assessment and Intervention, 10.1080/17489539.2020.1732024, (1-22), (2020).
  • Using Bayesian methods to test mediators of intervention outcomes in single-case experimental designs, Evidence-Based Communication Assessment and Intervention, 10.1080/17489539.2020.1732029, (1-17), (2020).
  • Fast talkers? Investigating the influence of self-talk on mental toughness and finish times in 800-meter runners, Journal of Applied Sport Psychology, 10.1080/10413200.2020.1735574, (1-19), (2020).
  • Investigating immediacy in multiple-phase-change single-case experimental designs using a Bayesian unknown change-points model, Behavior Research Methods, 10.3758/s13428-020-01345-z, (2020).
  • Acceptance and Commitment Therapy Focused on Repetitive Negative Thinking for Child Depression: A Randomized Multiple-Baseline Evaluation, The Psychological Record, 10.1007/s40732-019-00362-5, (2020).
  • A meta-analysis on the effectiveness of phonics instruction for teaching decoding skills to students with intellectual disability, Educational Research Review, 10.1016/j.edurev.2019.01.001, (2019).
  • Fitting Bayesian Models for Single-Case Experimental Designs, Methodology, 10.1027/1614-2241/a000180, 15, 4, (147-156), (2019).
  • Play to Teach: Coaching Paraeducators to Facilitate Communication in the Preschool Classroom, American Journal on Intellectual and Developmental Disabilities, 10.1352/1944-7558-124.6.497, 124, 6, (497-510), (2019).
  • A multiple randomization testing procedure for level, trend, variability, overlap, immediacy, and consistency in single-case phase designs, Behaviour Research and Therapy, 10.1016/j.brat.2019.103414, (103414), (2019).
  • Comparison of visual analysis, non-overlap methods, and effect sizes in the evaluation of parent implemented functional assessment based interventions, Research in Developmental Disabilities, 10.1016/j.ridd.2018.11.001, 85, (31-41), (2019).
  • Making Sense of Single‐Case Design Effect Sizes, Learning Disabilities Research & Practice, 10.1111/ldrp.12204, 34, 3, (124-132), (2019).
  • The use of touch devices for enhancing academic achievement: A meta‐analysis, Psychology in the Schools, 10.1002/pits.22225, 56, 7, (1187-1206), (2019).
  • Social Studies Content Knowledge Interventions for Students With Emotional and Behavioral Disorders: A Meta-Analysis, Behavior Modification, 10.1177/0145445519834622, (014544551983462), (2019).
  • A Meta-Analysis of Single-Case Research Using Mathematics Manipulatives With Students At Risk or Identified With a Disability, The Journal of Special Education, 10.1177/0022466919844516, (002246691984451), (2019).
  • An Evidence-Based Review and Meta-Analysis of Active Supervision, Behavioral Disorders, 10.1177/0198742919851021, (019874291985102), (2019).
  • We Don’t Train in Vain: A Systematic Review and Meta-Analysis of Human and Canine Caregiver Training, Journal of Applied Animal Welfare Science, 10.1080/10888705.2019.1646134, (1-37), (2019).
  • Game-Based Self-Management: Addressing Inattention during Independent Reading and Written Response, Journal of Applied School Psychology, 10.1080/15377903.2019.1660748, (1-24), (2019).
  • New approaches for estimation of effect sizes and their confidence intervals for treatment effects from randomized controlled trials, The Quantitative Methods for Psychology, 10.20982/tqmp.15.2.p096, 15, 2, (96-111), (2019).
  • A Systematic Review of the Evidence Base for Active Supervision in PK–12 Settings, Behavioral Disorders, 10.1177/0198742919837646, (019874291983764), (2019).
  • Individualized Instructional Delivery Options: Adapting Technology-Based Interventions for Students With Attention Difficulties, Journal of Special Education Technology, 10.1177/0162643419852929, (016264341985292), (2019).
  • Evaluation of a Home-Based Behavioral Treatment Model for Children With Tube Dependency, Journal of Pediatric Psychology, 10.1093/jpepsy/jsz014, (2019).
  • Teacher-Delivered Strategies to Increase Students’ Opportunities to Respond: A Systematic Methodological Review, Behavioral Disorders, 10.1177/0198742919828310, (019874291982831), (2019).
  • Bias Adjustment in Multilevel Meta-Analysis of Standardized Single-Case Experimental Data, The Journal of Experimental Education, 10.1080/00220973.2019.1658568, (1-18), (2019).
  • Mindfulness Interventions for College Students with ADHD: A Multiple Single Case Research Design, Journal of College Student Psychotherapy, 10.1080/87568225.2018.1450107, 33, 3, (199-220), (2018).
  • Comparing Visual and Statistical Analysis of Multiple Baseline Design Graphs, Behavior Modification, 10.1177/0145445518768723, 43, 3, (361-388), (2018).
  • A Systematic Review of Teacher-Delivered Behavior-Specific Praise on K–12 Student Performance, Remedial and Special Education, 10.1177/0741932517751054, 40, 2, (112-128), (2018).
  • Statistical analysis in Small-N Designs: using linear mixed-effects modeling for evaluating intervention effectiveness, Aphasiology, 10.1080/02687038.2018.1454884, 33, 1, (1-30), (2018).
  • Evaluating treatment outcomes for young people participating in a high‐intensity therapeutic violence intervention in the English Youth Custody Service, Legal and Criminological Psychology, 10.1111/lcrp.12142, 24, 1, (162-178), (2018).
  • A meta‐analytic review of the evidence for check‐in check‐out, Psychology in the Schools, 10.1002/pits.22195, 56, 3, (393-412), (2018).
  • A multiple-baseline evaluation of a brief acceptance and commitment therapy protocol focused on repetitive negative thinking for moderate emotional disorders, Journal of Contextual Behavioral Science, 10.1016/j.jcbs.2018.04.004, 9, (1-14), (2018).
  • A Gradual Effects Model for Single-Case Designs, Multivariate Behavioral Research, 10.1080/00273171.2018.1466681, 53, 4, (574-593), (2018).
  • Contingency Management Programs in Corrections: Another Panacea?, Journal of Contemporary Criminal Justice, 10.1177/1043986217750426, 34, 1, (35-46), (2018).
  • Between-case standardized effect size analysis of single case designs: Examination of the two methods, Research in Developmental Disabilities, 10.1016/j.ridd.2018.05.009, 79, (88-96), (2018).
  • Mediation analysis with binary outcomes: Direct and indirect effects of pro-alcohol influences on alcohol use disorders, Addictive Behaviors, 10.1016/j.addbeh.2018.12.018, (2018).
  • Development of a Distance-Based Effect Size Metric for Single-Case Research: Ratio of Distances, Behavior Therapy, 10.1016/j.beth.2018.02.005, 49, 6, (981-994), (2018).
  • Special Issue on Advances in Single-Case Research Design and Analysis, Developmental Neurorehabilitation, 10.1080/17518423.2018.1468600, 21, 4, (209-211), (2018).
  • Review of Single Subject Research Examining the Effectiveness of Interventions for At‐Risk English Learners, Learning Disabilities Research & Practice, 10.1111/ldrp.12160, 33, 2, (64-74), (2018).
  • Professional development to increase teacher behavior‐specific praise: A single‐case design replication, Psychology in the Schools, 10.1002/pits.22106, 55, 3, (264-277), (2018).
  • A summary of reporting guidelines and evaluation domains for using single‐case experimental designs and recommendations for the study of eating disorders, International Journal of Eating Disorders, 10.1002/eat.22887, 51, 7, (617-628), (2018).
  • The Positive Plus Program: Affirmative classroom management to improve student behavior, Behavioral Interventions, 10.1002/bin.1632, 33, 3, (221-236), (2018).
  • Protocol for a Systematic Review: Interventions for Anxiety in School‐Aged Children with Autism Spectrum Disorder (ASD): A Mixed‐Methods Systematic Review, Campbell Systematic Reviews, 10.1002/CL2.217, 14, 1, (1-48), (2018).
  • A Meta-Analysis of Interventions to Promote Self-Determination for Students With Disabilities, Remedial and Special Education, 10.1177/0741932518802274, (074193251880227), (2018).
  • Systematic Review of High Probability Requests in K-12 Settings: Examining the Evidence Base, Behavioral Disorders, 10.1177/0198742918800029, (019874291880002), (2018).
  • Time-Varying Effect Sizes for Quadratic Growth Models in Multilevel and Latent Growth Modeling, Structural Equation Modeling: A Multidisciplinary Journal, 10.1080/10705511.2018.1547110, (1-12), (2018).
  • Nonparametric meta-analysis for single-case research: Confidence intervals for combined effect sizes, Behavior Research Methods, 10.3758/s13428-018-1044-5, (2018).
  • The effectiveness of visual activity schedules for individuals with intellectual disabilities: A meta-analysis, Journal of Intellectual & Developmental Disability, 10.3109/13668250.2018.1431761, (1-12), (2018).
  • Response Cards to Increase Engagement and Active Participation of Middle School Students With EBD, Remedial and Special Education, 10.1177/0741932518800807, (074193251880080), (2018).
  • Randomized single-case AB phase designs: Prospects and pitfalls, Behavior Research Methods, 10.3758/s13428-018-1084-x, (2018).
  • Exploring the Effects of a Single Rational Emotive Behavior Therapy Workshop in Elite Blind Soccer Players, The Sport Psychologist, 10.1123/tsp.2017-0122, (1-12), (2018).
  • Approaches for Specifying the Level-1 Error Structure When Synthesizing Single-Case Data, The Journal of Experimental Education, 10.1080/00220973.2017.1409181, 87, 1, (55-74), (2017).
  • Systematic Use of Visual Analysis for Assessing Outcomes in Single Case Design Studies, Brain Impairment, 10.1017/BrImp.2017.16, 19, 1, (4-17), (2017).
  • A Meta-analysis of the Taped Problems Intervention, Journal of Behavioral Education, 10.1007/s10864-017-9284-5, 27, 1, (53-80), (2017).
  • The conditional power of randomization tests for single-case effect sizes in designs with randomized treatment order: A Monte Carlo simulation study, Behavior Research Methods, 10.3758/s13428-017-0885-7, 50, 2, (557-575), (2017).
  • Introduction to the Special Issue on Single-Case Systematic Reviews and Meta-Analyses, Remedial and Special Education, 10.1177/0741932517717043, 38, 6, (323-330), (2017).
  • Mindfulness-based interventions with youth: A comprehensive meta-analysis of group-design studies, Journal of School Psychology, 10.1016/j.jsp.2017.03.006, 63, (77-103), (2017).
  • Recommendations for Choosing Single-Case Data Analytical Techniques, Behavior Therapy, 10.1016/j.beth.2016.04.008, 48, 1, (97-114), (2017).
  • Effects of a Mindful Breathing Exercise During Reading Fluency Intervention for Students with Attentional Difficulties, Contemporary School Psychology, 10.1007/s40688-017-0132-3, 21, 4, (323-334), (2017).
  • A Meta-Analysis of Class-Wide Interventions for Supporting Student Behavior, School Psychology Review, 10.17105/SPR-2017-0015.V46-2, 46, 2, (149-164), (2017).
  • Functional Assessment–Based Interventions for Students With or At-Risk for High-Incidence Disabilities: Field Testing Single-Case Synthesis Methods, Remedial and Special Education, 10.1177/0741932517693320, 38, 6, (331-352), (2017).
  • Single-Case Design, Analysis, and Quality Assessment for Intervention Research, Journal of Neurologic Physical Therapy, 10.1097/NPT.0000000000000187, 41, 3, (187-197), (2017).
  • Technology-Aided Instruction and Intervention for Students With ASD: A Meta-Analysis Using Novel Methods of Estimating Effect Sizes for Single-Case Research, Remedial and Special Education, 10.1177/0741932517729508, 38, 6, (371-386), (2017).
  • A Meta-Analysis of School-Based Group Contingency Interventions for Students With Challenging Behavior: An Update, Remedial and Special Education, 10.1177/0741932517716900, 38, 6, (353-370), (2017).
  • Mindfulness‐based interventions for improving cognition, academic achievement, behavior, and socioemotional functioning of primary and secondary school students, Campbell Systematic Reviews, 10.4073/CSR.2017.5, 13, 1, (1-144), (2017).
  • Empirical and narrative approaches to case study designs: A commentary on ‘Nidotherapy in the successful management of comorbid depressive and personality disorder’, Personality and Mental Health, 10.1002/pmh.1386, 11, 4, (356-360), (2017).
  • A demonstration of how to do a meta-analysis that combines single-case designs with between-groups experiments: The effects of choice making on challenging behaviors performed by people with disabilities, Developmental Neurorehabilitation, 10.3109/17518423.2015.1100690, 21, 4, (266-278), (2016).
  • Incorporating Vocabulary Instruction in Individual Reading Fluency Interventions With English Language Learners, Canadian Journal of School Psychology, 10.1177/0829573516658855, 33, 1, (63-81), (2016).
  • Quantifying differences between conditions in single-case designs: Possible analysis and meta-analysis, Developmental Neurorehabilitation, 10.3109/17518423.2015.1100688, 21, 4, (238-252), (2016).
  • Single-case design and evaluation in R: An introduction and tutorial for school psychologists, International Journal of School & Educational Psychology, 10.1080/21683603.2016.1173610, 5, 1, (39-51), (2016).
  • Effects of a Summarizing Strategy on Written Summaries of Children With Emotional and Behavioral Disorders, Remedial and Special Education, 10.1177/0741932516669051, 38, 2, (87-97), (2016).
  • How Can Single-Case Data Be Analyzed? Software Resources, Tutorial, and Reflections on Analysis, Behavior Modification, 10.1177/0145445516664307, 41, 2, (179-228), (2016).
  • A metacognitive training pilot study for adolescents with autism spectrum disorder: Lessons learned from the preliminary stages of intervention development, Journal of Intellectual & Developmental Disability, 10.3109/13668250.2016.1210104, 42, 2, (204-210), (2016).
  • Confidence intervals for single-case effect size measures based on randomization test inversion, Behavior Research Methods, 10.3758/s13428-016-0714-4, 49, 1, (363-381), (2016).
  • Single-case experimental design yielded an effect estimate corresponding to a randomized controlled trial, Journal of Clinical Epidemiology, 10.1016/j.jclinepi.2016.01.035, 76, (82-88), (2016).
  • N-of-1 trials can be aggregated to generate group mean treatment effects: a systematic review and meta-analysis, Journal of Clinical Epidemiology, 10.1016/j.jclinepi.2016.03.026, 76, (65-75), (2016).
  • Seeing red: Relearning to read in a case of Balint’s Syndrome, NeuroRehabilitation, 10.3233/NRE-161342, 39, 1, (111-117), (2016).
  • Mindfulness-Based Exposure Strategies as a Transdiagnostic Mechanism of Change: An Exploratory Alternating Treatment Design, Behavior Therapy, 10.1016/j.beth.2015.10.008, 47, 2, (225-238), (2016).
  • Examining the Use of Deep Touch Pressure to Improve the Educational Performance of Students With Disabilities, Research and Practice for Persons with Severe Disabilities, 10.1177/1540796915624889, 41, 1, (3-18), (2016).
  • Does a pre-intervention functional assessment increase intervention effectiveness? A meta-analysis of within-subject interrupted time-series studies, Clinical Psychology Review, 10.1016/j.cpr.2016.05.003, 47, (71-84), (2016).
  • A survey of publication practices of single‐case design researchers when treatments have small or large effects, Journal of Applied Behavior Analysis, 10.1002/jaba.308, 49, 3, (656-673), (2016).
  • Between‐case standardized mean difference effect sizes for single‐case designs: a primer and tutorial using the scdhlm web application, Campbell Systematic Reviews, 10.4073/cmdp.2016.1, 12, 1, (1-31), (2016).
  • A Meta-Analysis of Video-Modeling Based Interventions for Reduction of Challenging Behaviors for Students With EBD, The Journal of Special Education, 10.1177/0022466915602493, 49, 4, (243-252), (2015).
  • Visual Attention and Key Word Sign in Children with Autism Spectrum Disorder, Journal of Developmental and Physical Disabilities, 10.1007/s10882-015-9443-3, 28, 1, (33-55), (2015).
  • The consequences of modeling autocorrelation when synthesizing single-case studies using a three-level model, Behavior Research Methods, 10.3758/s13428-015-0612-1, 48, 2, (803-812), (2015).
  • Further developments in summarising and meta-analysing single-case data: An illustration with neurobehavioural interventions in acquired brain injury, Neuropsychological Rehabilitation, 10.1080/09602011.2015.1064452, 25, 5, (637-662), (2015).
  • A Systematic Review of Meta-Analyses of Psychosocial Treatment for Attention-Deficit/Hyperactivity Disorder, Clinical Child and Family Psychology Review, 10.1007/s10567-015-0178-6, 18, 1, (77-97), (2015).
  • Comparing the percentage of non-overlapping data approach and the hierarchical linear modeling approach for synthesizing single-case studies in autism research, Research in Autism Spectrum Disorders, 10.1016/j.rasd.2014.12.002, 11, (112-125), (2015).
  • Comparing Visual and Statistical Analysis in Single-Case Studies Using Published Studies, Multivariate Behavioral Research, 10.1080/00273171.2014.973989, 50, 2, (162-183), (2015).
  • Computing Tools for Implementing Standards for Single-Case Designs, Behavior Modification, 10.1177/0145445515603706, 39, 6, (835-869), (2015).
  • Probability and Visual Aids for Assessing Intervention Effectiveness in Single-Case Designs, Behavior Modification, 10.1177/0145445515593512, 39, 5, (691-720), (2015).
  • PROTOCOL: Mindfulness‐Based Interventions for Improving Academic Achievement, Behavior and Socio‐Emotional Functioning of Primary and Secondary Students: A Systematic Review, Campbell Systematic Reviews, 10.1002/CL2.143, 11, 1, (1-47), (2015).
  • See more

The full text of this article hosted at iucr.org is unavailable due to technical difficulties.