Anthropogenic acid deposition may lead to soil acidification, with soil buffering capacity regulating the magnitude of any soil pH change. However, little evidence is available from large-scale observations. Here, we evaluated changes in soil pH across northern China's grasslands over the last two decades using soil profiles obtained from China's Second National Soil Inventory during the 1980s and a more recent regional soil survey during 2001–2005. A transect from the central-southern Tibetan Plateau to the eastern Inner Mongolian Plateau, where Kriging interpolation provided robust predictions of the spatial distribution of soil pH, was then selected to examine pH changes during the survey period. Our results showed that soil pH in the surface layer had declined significantly over the last two decades, with an overall decrease of 0.63 units (95% confidence interval = 0.54–0.73 units). The decline of soil pH was observed in both alpine grasslands on the Tibetan Plateau and temperate grasslands on the Inner Mongolian Plateau. Soil pH decreased more intensively in low soil carbonate regions, while changes of soil pH showed no significant associations with soil cation exchange capacity. These results suggest that grassland soils across northern China have experienced significant acidification from the 1980s to 2000s, with soil carbonates buffering the increase in soil acidity. The buffering process may induce a large loss of carbon from soil carbonates and thus alter the carbon balance in these globally important ecosystems.