Through 2–3-year (2003–2005) continuous eddy covariance measurements of carbon dioxide and water vapor fluxes, we examined the seasonal, inter-annual, and inter-ecosystem variations in the ecosystem-level water use efficiency (WUE, defined as the ratio of gross primary production, GPP, to evapotranspiration, ET) at four Chinese grassland ecosystems in the Qinghai-Tibet Plateau and North China. Representing the most prevalent grassland types in China, the four ecosystems are an alpine swamp meadow ecosystem, an alpine shrub-meadow ecosystem, an alpine meadow-steppe ecosystem, and a temperate steppe ecosystem, which illustrate a water availability gradient and thus provide us an opportunity to quantify environmental and biological controls on ecosystem WUE at different spatiotemporal scales. Seasonally, WUE tracked closely with GPP at the four ecosystems, being low at the beginning and the end of the growing seasons and high during the active periods of plant growth. Such consistent correspondence between WUE and GPP suggested that photosynthetic processes were the dominant regulator of the seasonal variations in WUE. Further investigation indicated that the regulations were mainly due to the effect of leaf area index (LAI) on carbon assimilation and on the ratio of transpiration to ET (T/ET). Besides, except for the swamp meadow, LAI also controlled the year-to-year and site-to-site variations in WUE in the same way, resulting in the years or sites with high productivity being accompanied by high WUE. The general good correlation between LAI and ecosystem WUE indicates that it may be possible to predict grassland ecosystem WUE simply with LAI. Our results also imply that climate change-induced shifts in vegetation structure, and consequently LAI may have a significant impact on the relationship between ecosystem carbon and water cycles in grasslands.