Haihe plain is an important food production area in China, facing an increasing water shortage. The water used for agriculture accounts for about 70% of total water resources. Thus, it is critical to optimize the irrigation scheduling for saving water and increasing crop water productivity (CWP). This study first simulated crop yield and CWP for winter wheat and summer maize in historical scenario during 1961–2005 for Haihe plain using previously well-established Soil and Water Assessment Tool model. Then, scenarios under historical irrigation (scenario 1) and sufficient irrigation (scenario 2) were, respectively, simulated both with sufficient fertilizer. The crop yield in scenario 2 was considered as the potential crop yield. The optimal irrigation scheduling with sufficient fertilizer (scenario 3) was explored by iteratively adjusting irrigation scheduling based on the scenario 1 and previous studies related to water stress on crop growth. Results showed that net irrigation amount was, respectively, reduced 23.1% and 18.8% in scenario 3 for winter wheat and summer maize when compared with scenario 1. The CWP was 12.1% and 8.2% higher with very slight change of crop yield. Using optimal irrigation scheduling could save 8.8 × 108 m3 irrigation water and reduce about 16.3% groundwater over-exploitation in winter wheat growth period. The corresponding yield was 18.5% and 12.9% less than potential yield for winter wheat and summer maize but using less irrigation water. Therefore, it could be considered that the optimal irrigation was reasonable, which provided beneficial suggestions for increasing efficiency of agricultural water use with sustainable crop yield in Haihe plain. Copyright © 2013 John Wiley & Sons, Ltd.