Research Article

Test–retest and between-site reliability in a multicenter fMRI study

  1. Lee Friedman1,*,
  2. Hal Stern2,
  3. Gregory G. Brown3,
  4. Daniel H. Mathalon4,
  5. Jessica Turner1,
  6. Gary H. Glover5,
  7. Randy L. Gollub6,7,
  8. John Lauriello8,
  9. Kelvin O. Lim9,
  10. Tyrone Cannon10,
  11. Douglas N. Greve7,
  12. Henry Jeremy Bockholt11,
  13. Aysenil Belger12,13,
  14. Bryon Mueller9,
  15. Michael J. Doty14,
  16. Jianchun He15,
  17. William Wells16,
  18. Padhraic Smyth17,
  19. Steve Pieper18,
  20. Seyoung Kim17,
  21. Marek Kubicki19,
  22. Mark Vangel6,20,
  23. Steven G. Potkin1

Article first published online: 17 JUL 2007

DOI: 10.1002/hbm.20440

Issue

Human Brain Mapping

Human Brain Mapping

Volume 29, Issue 8, pages 958–972, August 2008

Additional Information

How to Cite

Friedman, L., Stern, H., Brown, G. G., Mathalon, D. H., Turner, J., Glover, G. H., Gollub, R. L., Lauriello, J., Lim, K. O., Cannon, T., Greve, D. N., Bockholt, H. J., Belger, A., Mueller, B., Doty, M. J., He, J., Wells, W., Smyth, P., Pieper, S., Kim, S., Kubicki, M., Vangel, M. and Potkin, S. G. (2008), Test–retest and between-site reliability in a multicenter fMRI study. Human Brain Mapping, 29: 958–972. doi: 10.1002/hbm.20440

Author Information

  1. 1

    Department of Psychiatry and Human Behavior, University of California Irvine, Irvine, California

  2. 2

    Department of Statistics, University of California Irvine, Irvine, California

  3. 3

    Psychology Service 116B, University of California San Diego, VA San Diego Healthcare System, San Diego, California

  4. 4

    Psychiatry Service 116A, VA Connecticut Healthcare System, Connecticut

  5. 5

    Department of Radiology, Stanford University, Stanford, California

  6. 6

    Department of Psychiatry, Massachusetts General Hospital, Charlestown, Massachusetts

  7. 7

    Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, Massachusetts

  8. 8

    Department of Psychiatry, University of New Mexico, Albuquerque, New Mexico

  9. 9

    Department of Psychiatry, University of Minnesota, Minneapolis, Minnesota

  10. 10

    Department of Psychology, Los Angeles, California

  11. 11

    Morphometry and Neuroinformatics Core, The MIND Institute, 1101 Yale Boulevard NE, Albuquerque, New Mexico

  12. 12

    Duke-UNC Brain Imaging and Analysis Center, Duke University Medical Center, Durham, North Carolina

  13. 13

    Department of Psychiatry, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina

  14. 14

    Biomedical Engineering Core, The MIND Institute, 1101 Yale Boulevard NE, Albuquerque, New Mexico

  15. 15

    Department of Psychiatry, University of Iowa Hospital and Clinic, Iowa City, Iowa

  16. 16

    Department of Radiology, Harvard Medical School and Brigham and Women's Hospital, Boston, Massachusetts

  17. 17

    Department of Computer Science, University of California – Irvine, Irvine, California

  18. 18

    Isomics, Inc., 203 Franklin Street, Cambridge, Massachusetts

  19. 19

    Laboratory of Neuroscience, Department of Psychiatry, Harvard Medical School, Brockton, Massachusetts

  20. 20

    MGH/MIT GCRC Biomedical Imaging Core, Charlestown, Massachusetts

*1312 Michael Hughes Dr. NE, Albuquerque, NM 87112, USA

Publication History

  1. Issue published online: 11 JUL 2008
  2. Article first published online: 17 JUL 2007
  3. Manuscript Accepted: 23 MAY 2007
  4. Manuscript Received: 20 JUN 2006

Funded by

  • National Center for Research Resources (NCRR)
  • National Institutes of Health (NIH). Grant Number: 1 U24 RR021992
  • The MIND Institute

SEARCH

SEARCH BY CITATION

ARTICLE TOOLS

View Full Article (HTML) Get PDF (250K)

Keywords:

  • test–retest;
  • reproducibility;
  • intraclass correlation coefficient;
  • multicenter;
  • FMRI

Abstract

In the present report, estimates of test–retest and between-site reliability of fMRI assessments were produced in the context of a multicenter fMRI reliability study (FBIRN Phase 1, www.nbirn.net). Five subjects were scanned on 10 MRI scanners on two occasions. The fMRI task was a simple block design sensorimotor task. The impulse response functions to the stimulation block were derived using an FIR-deconvolution analysis with FMRISTAT. Six functionally-derived ROIs covering the visual, auditory and motor cortices, created from a prior analysis, were used. Two dependent variables were compared: percent signal change and contrast-to-noise-ratio. Reliability was assessed with intraclass correlation coefficients derived from a variance components analysis. Test–retest reliability was high, but initially, between-site reliability was low, indicating a strong contribution from site and site-by-subject variance. However, a number of factors that can markedly improve between-site reliability were uncovered, including increasing the size of the ROIs, adjusting for smoothness differences, and inclusion of additional runs. By employing multiple steps, between-site reliability for 3T scanners was increased by 123%. Dropping one site at a time and assessing reliability can be a useful method of assessing the sensitivity of the results to particular sites. These findings should provide guidance toothers on the best practices for future multicenter studies. Hum Brain Mapp, 2008. © 2007 Wiley-Liss, Inc.

View Full Article (HTML) Get PDF (250K)