Low 25-hydroxyvitamin D (VitD), low sex hormones (SH), and high sex hormone binding globulin (SHBG) levels are common in older men. We tested the hypothesis that combinations of low VitD, low SH, and high SHBG would have a synergistic effect on bone mineral density (BMD), bone loss, and fracture risk in older men. Participants were a random subsample of 1468 men (mean age 74 years) from the Osteoporotic Fractures in Men Study (MrOS) plus 278 MrOS men with incident nonspine fractures studied in a case-cohort design. “Abnormal” was defined as lowest quartile for VitD (<20 ng/mL), bioavailable testosterone (BioT, <163 ng/dL), and bioavailable estradiol (BioE, <11 pg/mL); and highest quartile for SHBG (>59 nM). Overall, 10% had isolated VitD deficiency; 40% had only low SH or high SHBG; 15% had both SH/SHBG and VitD abnormality; and 35% had no abnormality. Compared to men with all normal levels, those with both SH/SHBG and VitD abnormality tended to be older, more obese, and to report less physical activity. Isolated VitD deficiency, and low BioT with or without low VitD, was not significantly related to skeletal measures. The combination of VitD deficiency with low BioE and/or high SHBG was associated with significantly lower baseline BMD and higher annualized rates of hip bone loss than SH abnormalities alone or no abnormality. Compared to men with all normal levels, the multivariate-adjusted hazard ratio (95% confidence interval [CI]) for incident nonspine fracture during 4.6-year median follow-up was 1.2 (0.8–1.8) for low VitD alone; 1.3 (0.9–1.9) for low BioE and/or high SHBG alone; and 1.6 (1.1–2.5) for low BioE/high SHBG plus low VitD. In summary, adverse skeletal effects of low sex steroid levels were more pronounced in older men with low VitD levels. The presence of low VitD in the presence of low BioE/high SHBG may contribute substantially to poor skeletal health. © 2012 American Society for Bone and Mineral Research.