Residual errors of hydrological models are usually both heteroscedastic and autocorrelated. However, only a few studies have attempted to explicitly include these two statistical properties into the residual error model and jointly infer them with the hydrological model parameters. This technical note shows that applying autoregressive error models to raw heteroscedastic residuals, as done in some recent studies, can lead to unstable error models with poor predictive performance. This instability can be avoided by applying the autoregressive process to standardized residuals. The theoretical analysis is supported by empirical findings in three hydrologically distinct catchments. The case studies also highlight strong interactions between the parameters of autoregressive residual error models and the water balance parameters of the hydrological model.