It is well recognized that improving nitrogen use efficiency (NUE) can directly reduce nitrous oxide (N2O) emission in cropland and indirectly reduce carbon dioxide (CO2) release from nitrogen (N) production, while such a reduction has not been well quantified in China. We estimated the greenhouse gas (GHG; N2O and CO2) mitigation potential (MP) from Chinese cropland and its regional distribution by quantifying NUE and determining the amount of over-applied synthetic N under various scenarios of NUE. We estimated that synthetic NUE in the late 1990s was 31±11% (mean±SD) for rice, 33±13% for wheat, and 31±11% for maize cultivation. Improving NUE to 50% could cut 6.6 Tg of synthetic N use per year, accounting for 41% of the total used. As a result of this reduction, the direct N2O emission from croplands together with CO2 emission from the industrial production and transport of synthetic N could be reduced by 39%, equivalent to 60 Tg CO2 yr−1. The MP was probably underestimated because organic N supply was not taken into account when estimating NUE. It was concluded that improving N management can greatly reduce GHG (N2O and CO2) emissions in Chinese croplands, and mitigation in the Jiangsu, Henan, Shandong, Sichuan, Hubei, Anhui, and Hebei provinces should be given priority.