The aim of this paper is to derive the principal components (PCs) in variations of (i) the solar background magnetic field (SBMF), measured by the Wilcox Solar Observatory with low spatial resolution for solar cycles 21–23, and (ii) the sunspot magnetic field (SMF) in cycle 23, obtained by SOHO/MDI. For reduction of the component dimensions, principal component analysis (PCA) is carried out to identify global patterns in the data and to detect pairs of PCs and corresponding empirical orthogonal functions (EOFs). PCA reveals two main temporal PCs in the SBMF of opposite polarities originating in opposite hemispheres and running noticeably off-phase (with a delay of about 2.5 yr), with their maxima overlapping in the most active hemisphere for a given cycle. Their maximum magnitudes are reduced by a factor of 3 from cycle 21 to 23, and overlap in the Northern hemisphere for cycle 21, in the Southern one in cycle 22 and in the Northern one again in cycle 23. The reduction of magnitudes and slopes of the maxima of the SBMF waves from cycle 21 to cycle 23 leads us to expect lower magnitudes of the SBMF wave in cycle 24. In addition, PCA allowed us to detect four pairs of EOFs in the SBMF latitudinal components: the two main latitudinal EOFs attributed to symmetric types and another three pairs of EOFs assigned to asymmetric types of meridional flows. The results allow us to postulate the existence of dipole and quadruple (or triple-dipole) magnetic structures in the SBMF, which vary from cycle to cycle and take the form of two waves travelling off-phase, with a phase shift of one-quarter of the 11 yr period. Similar PC and EOF components were found in temporal and latitudinal distributions of the SMF for cycle 23, revealing polarities opposite to the SBMF polarities, and a double maximum in time or maxima in latitude corresponding to the maxima of the SBMF PC residuals or minima in the SBMF EOFs, respectively. This suggests that the SBMF waves modulate the occurrence and magnitude of the SMF in time and latitude.