It is not known how the profoundly complex topography and habitat heterogeneity generated by the uplift of the Qinghai-Tibetan Plateau (QTP) during the late Tertiary affected population genetic structure of endangered Taxus yunnanensis. In addition, the effects of habitat fragmentation due to anthropogenic disturbance on genetic diversity and population differentiation of this species have not been studied. T. yunnanensis is an ancient tree/shrub mainly distributed in southwest China. Recently, the species has suffered a sharp decline due to excessive logging for its famous anticancer metabolite taxol, resulting in smaller and more isolated populations. To understand the phylogeography and genetic consequences of habitat fragmentation of this endangered species, using 11 polymorphic microsatellites, we genotyped 288 individuals from 14 populations from a range-wide sampling in China. Our results suggest that two different population groups that were once isolated have persisted in situ during glacial periods in both areas, and have not merged since. Habitat fragmentation has led to significant genetic bottlenecks, high inbreeding and population divergence in this species. The two different population groups of T. yunnanensis could be attributed to restricted gene flow caused through isolation by geographical barriers and by habitat heterogeneity during uplift of the QTP, or the existence of two separate glacial refugia during the Pleistocene. In situ and ex situ conservation of the two Evolutionarily Significant Units (ESUs), artificial gene flow between populations and a comprehensive understanding of the pollination system in this endangered species are suggested from this study.