Although a link between body mass index (BMI) and brain volume has been established in several cross-sectional studies, evidence of the association between change in BMI over time and changes in brain structure is limited. Using data from a cohort of 347 former lead workers and community controls with two magnetic resonance imaging scans over a period of ∼5 years, we estimated cross-sectional and longitudinal associations of BMI and brain volume using both region of interest (ROI) and voxel-based morphometric (VBM) methods. We found that associations of BMI and brain volume were not significantly different in former lead workers when compared with community controls. In the cross-sectional analysis, higher BMIs were associated with smaller brain volumes in gray matter (GM) using both ROI and VBM approaches. No associations with white matter (WM) were observed. In the longitudinal analysis, higher baseline BMI was associated with greater decline in temporal and occipital GM ROI volumes. Change in BMI over the 5-year period was only associated with change in hippocampal volume and was not associated with change in any of the GM ROIs. Overall, higher BMI was associated with lower GM volume in several ROIs and with declines in volume in temporal and occipital GM over time. These results suggest that sustained high body mass may contribute to progressive temporal and occipital atrophy. Hum Brain Mapp 35:75–88, 2014. © 2012 Wiley Periodicals, Inc.