Global surface temperature is predicted to increase by 1.4–5.8°C by the end of this century. However, the impacts of this projected warming on soil C balance and the C budget of terrestrial ecosystems are not clear. One major source of uncertainty stems from warming effects on soil microbes, which exert a dominant influence on the net C balance of terrestrial ecosystems by controlling organic matter decomposition and plant nutrient availability. We, therefore, conducted an experiment in a tallgrass prairie ecosystem at the Great Plain Apiaries (near Norman, OK) to study soil microbial responses to temperature elevation of about 2°C through artificial heating in clipped and unclipped field plots. While warming did not induce significant changes in net N mineralization, soil microbial biomass and respiration rate, it tended to reduce extractable inorganic N during the second and third warming years, likely through increasing plant uptake. In addition, microbial substrate utilization patterns and the profiles of microbial phospholipid fatty acids (PLFAs) showed that warming caused a shift in the soil microbial community structure in unclipped subplots, leading to the relative dominance of fungi as evidenced by the increased ratio of fungal to bacterial PLFAs. However, no warming effect on soil microbial community structure was found in clipped subplots where a similar scale of temperature increase occurred. Clipping also significantly reduced soil microbial biomass and respiration rate in both warmed and unwarmed plots. These results indicated that warming-led enhancement of plant growth rather than the temperature increase itself may primarily regulate soil microbial response. Our observations show that warming may increase the relative contribution of fungi to the soil microbial community, suggesting that shifts in the microbial community structure may constitute a major mechanism underlying warming acclimatization of soil respiration.