Correlated Component Analysis (CCA) allows us to estimate how the different diffuse emissions mix in observations by cosmic microwave background (CMB) experiments, also taking into account complementary information from other surveys. It is especially useful for dealing with possible additional components for which little or no prior information exists. An application of CCA to the Wilkinson Microwave Anisotropy Probe (WMAP) maps assuming that only the canonical Galactic emissions (synchrotron, free–free and thermal dust) are present highlights the widespread presence of a spectrally flat ‘synchrotron’ component, largely uncorrelated with the synchrotron template, suggesting that an additional foreground is indeed required. We have tested various spectral shapes for such a component, namely a power law as is expected if it is flat synchrotron, and two spectral shapes that may fit the spinning dust emission: a parabola in the log S− log ν plane and a grey body. If the spatial distribution of the additional (‘anomalous’) component is not constrained a priori, it is found to be always tightly correlated with thermal dust, but the correlation is not perfect. Quality tests applied to the reconstructed CMB maps clearly disfavour two of the models. The CMB power spectra, estimated from CMB maps reconstructed exploiting the three surviving foreground models, are generally consistent with those obtained by the WMAP team, although at least one of the models gives a significantly higher quadrupole moment than found by WMAP. Taking foreground modelling uncertainties into account, we find that the mean quadrupole amplitude for the three ‘good’ models is less than 1σ below the expectation from the standard Λcold dark matter (ΛCDM) model. Also, the other reported deviations from model predictions are found not to be statistically significant, except for the excess power at l≃ 40. We confirm the evidence for a marked north–south asymmetry in the large-scale (l < 20) CMB anisotropies, which is stable with respect to the foreground parametrization we adopted. We also present a first, albeit preliminary, all-sky template of the ‘anomalous emission’.