Risk adjustment of heroin treatment outcomes for comparative performance assessment in England
Article first published online: 10 AUG 2012
© 2012 The Authors, Addiction © 2012 Society for the Study of Addiction
Volume 107, Issue 12, pages 2161–2172, December 2012
How to Cite
Marsden, J., Eastwood, B., Jones, H., Bradbury, C., Hickman, M., Knight, J., Randhawa, K., White, M. and the National Drug Treatment Monitoring System Outcomes Study Group (2012), Risk adjustment of heroin treatment outcomes for comparative performance assessment in England. Addiction, 107: 2161–2172. doi: 10.1111/j.1360-0443.2012.03971.x
- Issue published online: 20 NOV 2012
- Article first published online: 10 AUG 2012
- Accepted manuscript online: 12 JUN 2012 06:31AM EST
- Manuscript Accepted: 29 MAY 2012
- Manuscript Revised: 16 MAR 2012
- Manuscript Received: 13 JAN 2012
- Comparative performance monitoring;
- heroin abstinence;
- heroin deterioration;
- reliable change index;
- risk adjustment;
- substance use disorder treatment;
- treatment outcomes
Variability in effectiveness of treatment for substance abuse disorder (SUD) is an important and understudied issue. This study aimed to quantify the extent of outcome variability in the English SUD treatment system after adjusting for potential confounding variables.
Prospective cohort study using data from the English national drug treatment outcome monitoring database.
All 149 administrative areas delivering publicly funded SUD services in the National Health Service and non-governmental sector.
New adult admissions between January 2008 and October 2010 with illicit heroin-related problems in all administrative areas, with an in-treatment review conducted between 5 and 26 weeks (mean = 129.5 days; SD = 40.0) up to 30 April 2011 (n = 65 223; 75.6% of eligible clients). Individuals were divided randomly to form model developmental and internal validation samples. These were contrasted with an independent (external) sample of the same population admitted to treatment between November 2010 and April 2011 and followed to 31 October 2011 (n = 13 797; 81.4% of those eligible).
Measurements and analysis
The outcome measure was self-reported illicit heroin use, categorized as abstinent or deteriorated (the latter by Reliable Change Index), each risk-adjusted by person-level (demographics, clinical severity and treatment complexity) and area-level (SUD prevalence, social deprivation and severity averages) covariates by multivariable logistic regression using multiply imputed outcome and covariate data. Risk-adjusted models were assessed by information criteria and discrimination (c-index). Standardized outcome rates were compared by funnel plot with 95% and 99% control limits.
Models of heroin abstinence (48.4%) and deterioration (3.2%) were comparable across the developmental and validation samples (c-index = 0.70–0.71 and 0.82–0.87), with 79.2 and 94.0%, respectively, of the 149 treatment areas falling within 95% control limits. At the 99% limit, seven areas (4.7%) achieved abstinence rates above the national average, and eight had relatively poor abstinence rates (5.4%). At the 99% control limit, one area achieved very low deterioration outcomes and two (1.3%) were worse that the average. Risk adjustment served to increase abstinence rates in good performing areas by 0.63% and reduce abstinence rates by 0.37% in poor performing areas, and by 0.12% and 0.18%, respectively, for deterioration.
There is some exceptional variability in the apparent effectiveness of the English treatment system for substance use disorders. It is important to determine the source of this variability in order to inform drug treatment delivery and its evaluation both in England and overseas.