Between-person differences in white matter microstructure may partly generalize across the brain and partly play out differently for distinct tracts. We used diffusion-tensor imaging and structural equation modeling to investigate this issue in a sample of 260 adults aged 60–87 years. Mean fractional anisotropy and mean diffusivity of seven white matter tracts in each hemisphere were quantified. Results showed good fit of a model positing that individual differences in white matter microstructure are structured according to tracts. A general factor, although accounting for variance in the measures, did not adequately represent the individual differences. This indicates the presence of a substantial amount of tract-specific individual differences in white matter microstructure. In addition, individual differences are to a varying degree shared between tracts, indicating that general factors also affect white matter microstructure. Age-related differences in white matter microstructure were present for all tracts. Correlations among tract factors did not generally increase as a function of age, suggesting that aging is not a process with homogenous effects on white matter microstructure across the brain. These findings highlight the need for future research to examine whether relations between white matter microstructure and diverse outcomes are specific or general. Hum Brain Mapp, 2013. © 2012 Wiley Periodicals, Inc.