A comparison of agency-based and self-report methods of measuring services across an urban environment by a drug-abusing homeless population
Article first published online: 20 MAR 2006
Copyright © 2006 John Wiley & Sons, Ltd.
International Journal of Methods in Psychiatric Research
Volume 15, Issue 1, pages 46–56, March 2006
How to Cite
Pollio, D. E., North, C. S., Eyrich, K. M., Foster, D. A. and Spitznagel, E. L. (2006), A comparison of agency-based and self-report methods of measuring services across an urban environment by a drug-abusing homeless population. Int. J. Methods Psychiatr. Res., 15: 46–56. doi: 10.1002/mpr.28
- Issue published online: 20 MAR 2006
- Article first published online: 20 MAR 2006
- service measurement;
- substance abuse;
The purpose of this paper is to advance the methodology for studying service assessment by comparing self-report and agency-generated methods. This study compares 30-day self-reported service use for homeless individuals (N = 229) randomly recruited from a single urban environment (St Louis, Missouri) with similar data collected from a broad array of service agencies providing homeless, substance abuse and outpatient mental health services across the same environment. Comparisons were made between self-report and agency-based data on shelter use, outpatient mental health service use (case management, psychiatric treatment, group therapy), outpatient substance abuse service use (case management, counselling, group therapy) and drop-in/day treatment use. Consistently low levels of kappa scores (all under 0.4) and correlation coefficients (only shelter use demonstrated significant agreement) were found. Findings demonstrated that the two methods of collecting service data are generally not concordant at the individual level. Certain demographic characteristics (increased age, being male, non-white ethnicity) and diagnoses (cocaine abuse/dependence, mania, schizophrenia) were associated with decreased reliability between the two methods of data collection. The two methods of assessment appeared to capture overlapping but not identical information. Each method of assessment has different utility to researchers and providers wishing to assess service use. Copyright © 2006 John Wiley & Sons, Ltd.