Our Earth is facing the challenge of accelerating climate change, which imposes a great threat to biodiversity. Many published studies suggest that climate warming may cause a dramatic decline in biodiversity, especially in colder and drier regions. In this study, we investigated the effects of temperature, precipitation and a normalized difference vegetation index on biodiversity indices of rodent communities in the current or previous year for both detrended and nondetrended data in semi-arid grassland of Inner Mongolia during 1982–2006. Our results demonstrate that temperature showed predominantly positive effects on the biodiversity of small rodents; precipitation showed both positive and negative effects; a normalized difference vegetation index showed positive effects; and cross-correlation function values between rodent abundance and temperature were negatively correlated with rodent abundance. Our results suggest that recent climate warming increased the biodiversity of small rodents by providing more benefits to population growth of rare or less abundant species than that of more abundant species in Inner Mongolia grassland, which does not support the popular view that global warming would decrease biodiversity in colder and drier regions. We hypothesized that higher temperatures might benefit rare or less abundant species (with smaller populations and more folivorous diets) by reducing the probability of local extinction and/or by increasing herbaceous food resources.