Droughts and floods are frequent disasters in the Yellow River basin in northern China. They have a strong impact on agriculture and water resource management. To explore the physical mechanisms of these droughts and floods, influences exerted by the sea surface temperature (SST) anomalies in the mid-latitude North Atlantic on the wetness and dryness in the Yellow River basin have been investigated. The drought/flood index (DFI), derived from a reconstructed dataset covering the last five centuries, is used. Numerical simulations are conducted with the Community Atmospheric Model version 3.0 (CAM3) to study the influence of various mechanisms. It is found that DFI in the entire Yellow River basin experienced oscillations at about 50–60 years during the past five centuries. Droughts and floods have inconsistent patterns in different areas of the Yellow River. The periodic variation of DFI in Xi'an, a station in a semi-humid subarea, is in contrast to the North Atlantic SST (10°N–55°N and 70°W–10°W) oscillation in spring and summer; the periodic variation of the Yinchuan DFI in a semi-arid subarea (the upper reaches of Yellow River) correlates with the North Atlantic SST oscillation variation in spring and summer. The North Atlantic SST is probably one of the key sources of internal variability of the climate system, which results in Rossby wave adjusting. The warm phase of the North Atlantic SST is related to the operation of a Northern Atlantic Oscillation (NAO) like pattern conducive to easterly wind anomalies in northern China and enhances anti-cyclones around Lake Baikal that leads to less precipitation or more frequent droughts in the semi-arid subarea in the upper reaches of the Yellow River, but wet conditions in the semi-humid subarea in the middle reaches, monsoon subarea in the lower reaches and plateau subarea in the upper reaches of the river.