Calorie restriction (CR) reduces bone quantity but not bone quality in rodents. Nothing is known regarding the long-term effects of CR with adequate intake of vitamin and minerals on bone quantity and quality in middle-aged lean individuals. In this study, we evaluated body composition, bone mineral density (BMD), and serum markers of bone turnover and inflammation in 32 volunteers who had been eating a CR diet (∼35% less calories than controls) for an average of 6.8 ± 5.2 years (mean age 52.7 ± 10.3 years) and 32 age- and sex-matched sedentary controls eating Western diets (WD). In a subgroup of 10 CR and 10 WD volunteers, we also measured trabecular bone (TB) microarchitecture of the distal radius using high-resolution magnetic resonance imaging. We found that the CR volunteers had significantly lower body mass index than the WD volunteers (18.9 ± 1.2 vs. 26.5 ± 2.2 kg m−2; P = 0.0001). BMD of the lumbar spine (0.870 ± 0.11 vs. 1.138 ± 0.12 g cm−2, P = 0.0001) and hip (0.806 ± 0.12 vs. 1.047 ± 0.12 g cm−2, P = 0.0001) was also lower in the CR than in the WD group. Serum C-terminal telopeptide and bone-specific alkaline phosphatase concentration were similar between groups, while serum C-reactive protein (0.19 ± 0.26 vs. 1.46 ± 1.56 mg L−1, P = 0.0001) was lower in the CR group. Trabecular bone microarchitecture parameters such as the erosion index (0.916 ± 0.087 vs. 0.877 ± 0.088; P = 0.739) and surface-to-curve ratio (10.3 ± 1.4 vs. 12.1 ± 2.1, P = 0.440) were not significantly different between groups. These findings suggest that markedly reduced BMD is not associated with significantly reduced bone quality in middle-aged men and women practicing long-term calorie restriction with adequate nutrition.