We have analysed a sample of 1292 4.5-μm-selected galaxies at z≥ 3, over 0.6 deg2 of the UKIRT Infrared Deep Survey (UKIDSS) Ultra Deep Survey (UDS). Using photometry from the U band through 4.5 μm, we have obtained photometric redshifts and derived stellar masses for our sources. Only two of our galaxies potentially lie at z > 5. We have studied the galaxy stellar mass function at 3 ≤z < 5, based on the 1213 galaxies in our catalogue with 4.5-μm magnitudes ≤24.0. We find that (i) the number density of galaxies increased by a factor of >10 between z= 5 and 3, indicating that the assembly rate of these galaxies proceeded >20 times faster at these redshifts than at 0 < z < 2; (ii) the Schechter function slope α is significantly steeper than that displayed by the local stellar mass function, which is a consequence of both the steeper faint end and the absence of a pure exponential decline at the high-mass end; and (iii) the evolution of the comoving stellar mass density from z= 0 to 5 can be modelled as log10ρM=−(0.05 ± 0.09)z2−(0.22 ∓ 0.32)z+ 8.69. At 3 ≤z < 4, more than 30 per cent of the galaxies would be missed by optical surveys with R < 27 or z < 26. Thus, our study demonstrates the importance of deep mid-infrared surveys over large areas to perform a complete census of massive galaxies at high z and trace the early stages of massive galaxy assembly.