Many species have experienced dramatic declines over the past millennia due to the accelerated impact of human activity and climate change, but compelling evidence over such long-term time scales is rare. China has a unique system archiving historical records of important social, meteorological, agricultural and biological events over the last three millennia. We derived historical species occurrences (0–2000 AD) based on a comprehensive review of literature. To detect the driving forces of range contraction, we used correlation and multiple regression to quantify the linear association between species range indices and climate variables (five temperature series and three precipitation series), as well as a human population size series. We also used a machine learning technique, random forest, to quantify the nonlinear effects of the climate variables and human population size. The southward retreat of the Asian elephant Elephas maximus and the rhinoceroses (Dicerorhinus sumatrensis, Rhinoceros unicornis, R. sondaicus) was closely associated with climate cooling and intensified human impact (represented by high population size), and the westward retreat of the giant panda Ailuropoda melanoleuca was associated with intensified human impact. One temperature series and human population size showed interactive effect on range shift of the Asian elephant and the rhinoceroses; the effect of temperature was positive at low population size, but negative at high population size. Our results imply that a higher temperature caused the northward or eastward range shift of the Asian elephant, the rhinoceroses and the giant panda, and currently this trend is impeded by human activities. We also illustrate how human activity and climate act synergistically to cause range contraction.