Understanding the impacts that influence water quality is critical to the development of best management practices at the large watershed scale. This study describes the spatiotemporal variation in surface water quality and identifies their main impact in the Haihe River basin, China. Multivariate statistical techniques are applied to analyze the similarities among the sampling sites and to identify the main pollution sources in surface water. Results show that: (1) the basin can be clustered into two regions, water quality being better in the mountainous vs. plain regions; (2) water quality improves due to implementation of a strict state policy on environmental pollution control, prodded by the hosting of the Olympic games in the cities of Beijing and Tianjin; and (3) agricultural and residential land uses as well as livestock-breeding are the main sources affecting water quality in the mountainous regions, whereas rural waste discharge — including domestic waste sewage, human and animal feces, and solid waste — significantly influences water quality in the plain regions. The waste discharge of industrial factories may be a significant source of water pollution in the plain regions. Results indicate that the environmental management from pollution sinks and sources, long-lasting legal framework, and adequate economic incentives should be improved to optimize the large-scale watershed management under the background of the rapid development of countries like China.