High bone mass in animals and humans with sclerostin deficiency is associated with increased bone strength, which is not the case for all disorders with high bone mineral density, some of which are even associated with fragility fractures owing to unfavorable bone composition. In the current study we investigated whether alterations in bone composition may contribute to the bone strength characteristics associated with lack of sclerostin. We examined cortical bone of Sost-knockout (KO) mice (n = 9, 16 weeks old) and sclerosteosis patients (young [4 to 14 years], n = 4 and adults [24 and 43 years], n = 2) by quantitative backscattered electron imaging and Raman microspectroscopy and compared it to bone from wild-type mice and healthy subjects, respectively. In Sost-KO mice endocortical bone exhibited altered bone composition, whereas subperiosteal bone was unchanged. When comparing endocortical bone tissue of identical tissue age as defined by sequential dual fluorochrome labeling the average bone matrix mineralization was reduced −1.9% (p < 0.0001, younger tissue age) and −1.5% (p < 0.05, older tissue age), and the relative proteoglycan content was significantly increased. Similarly, bone matrix mineralization density distribution was also shifted toward lower matrix mineralization in surgical samples of compact bone of sclerosteosis patients. This was associated with an increase in mineralization heterogeneity in the young population. In addition, and consistently, the relative proteoglycan content was increased. In conclusion, we observed decreased matrix mineralization and increased relative proteoglycan content in bone subcompartments of Sost-KO mice—a finding that translated into sclerosteosis patients. We hypothesize that the altered bone composition contributes to the increased bone strength of patients with sclerostin deficiency. © 2014 American Society for Bone and Mineral Research.