This study presents single-objective and multi-objective particle swarm optimization (PSO) algorithms for automatic calibration of Hydrologic Engineering Center- Hydrologic Modeling Systems rainfall-runoff model of Tamar Sub-basin of Gorganroud River Basin in north of Iran. Three flood events were used for calibration and one for verification. Four performance criteria (objective functions) were considered in multi-objective calibration where different combinations of objective functions were examined. For comparison purposes, a fuzzy set-based approach was used to determine the best compromise solutions from the Pareto fronts obtained by multi-objective PSO. The candidate parameter sets determined from different single-objective and multi-objective calibration scenarios were tested against the fourth event in the verification stage, where the initial abstraction parameters were recalibrated. A step-by-step screening procedure was used in this stage while evaluating and comparing the candidate parameter sets, which resulted in a few promising sets that performed well with respect to at least three of four performance criteria. The promising sets were all from the multi-objective calibration scenarios which revealed the outperformance of the multi-objective calibration on the single-objective one. However, the results indicated that an increase of the number of objective functions did not necessarily lead to a better performance as the results of bi-objective function calibration with a proper combination of objective functions performed as satisfactorily as those of triple-objective function calibration. This is important because handling multi-objective optimization with an increased number of objective functions is challenging especially from a computational point of view. Copyright © 2012 John Wiley & Sons, Ltd.