Precipitation pulses play an important role in regulating ecosystem carbon exchange and balance of semiarid steppe ecosystems. It has been predicted that the frequency of extreme rain events will increase in the future, especially in the arid and semiarid regions. We hypothesize that large rain pulses favor carbon sequestration, while small ones cause more carbon release in the semiarid steppes. To understand the potential response in carbon sequestration capacity of semiarid steppes to the changes in rain pulse size, we conducted a manipulative experiment with five simulated rain pulse sizes (0, 5, 10, 25, and 75 mm) in Inner Mongolia steppe. Our results showed that both gross ecosystem productivity (GEP) and ecosystem respiration (Re) responded rapidly (within 24 h) to rain pulses and the initial response time was independent of pulse size. However, the time of peak GEP was 1–3 days later than that of Re, which depended on pulse size. Larger pulses caused greater magnitude and longer duration of variations in GEP and Re. Differences in the response time of microbes and plants to wetting events constrained the response pattern of heterotrophic (Rh) and autotrophic (Ra) components of Re following a rain event. Rh contributed more to the increase of Re in the early stage of rain pulse response, while Ra played an more important role later, and determined the duration of pulse response, especially for large rain events of >10 mm. The distinct responses of ecosystem photosynthesis and respiration to increasing pulse sizes led to a threshold in rain pulse size between 10 and 25 mm, above which post wetting responses favored carbon sequestration. The disproportionate increase of the primary productivity of higher plants, compared with those in the activities of microbial decomposers to larger pulse events suggests that the carbon sequestration capacity of Inner Mongolia steppes will be sensitive to changes in precipitation size distribution rather than just precipitation amount.