Soil carbon (C) stocks consist of inorganic and organic components, ~1.7 times larger than the total of the C stored in vegetation and the atmosphere together. Significant soil C losses could thus offset any C sink in vegetation, creating a positive feedback to climate change. However, compared with the susceptible sensitivity of organic matter decay to climate warming, soil inorganic carbon (SIC) stocks are often assumed to be relatively stable. Here, we evaluated SIC changes across China's grasslands over the last two decades using data from a recent regional soil survey during 2001–2005 and historical national soil inventory during the 1980s. Our results showed that SIC stocks in the top 10 cm decreased significantly between the two sampling periods, with a mean rate of 26.8 (95% confidence interval: 15.8–41.7) g C m−2 yr−1. The larger decreases in SIC stocks were observed in those regions with stronger soil acidification and richer soil carbonates. The lost SIC could be released to the atmosphere as carbon dioxide, redistributed to the deeper soil layer, and transferred to the nearby regions. The fraction of soil carbonates entering into the atmosphere may diminish the strength of terrestrial C sequestration and amplify the positive C-climate feedback.