The eco-hydrological model SWIM was used to examine the effects of forestation on water yield in a watershed of the Liupan Mountains in northwest China. The results showed that the water yield variation caused by tree species shift among mature forests dominated by larch, poplar and birch was negligible. The vegetation type conversion from grassland to forest strongly reduced water yield. The annual water yield reduction after 10% forestation was 15.8 mm on average with a fluctuation from 3.5 to 19.3 mm. The contribution of site variation to water yield varied from a decrease of 3.5 mm to an increase of 12.3 mm after 10% forestation, which on average was nearly a half of the influence of vegetation conversion between forest and grassland. Site selection for forestation in mountainous areas could be beneficial in alleviating forest-water conflicts and lessening the water yield reduction caused by forestation.