Millisecond pulsars are mainly characterized by their spin periods, B-fields and masses – quantities that are largely affected by previous interactions with a companion star in a binary system. In this paper, we investigate the formation mechanism of millisecond pulsars by considering the pulsar recycling process in both intermediate-mass X-ray binaries (IMXBs) and low-mass X-ray binaries (LMXBs). The IMXBs mainly lead to the formation of binary millisecond pulsars with a massive carbon–oxygen (CO) or an oxygen–neon–magnesium white dwarf (ONeMg WD) companion, whereas the LMXBs form recycled pulsars with a helium white dwarf (He WD) companion. We discuss the accretion physics leading to the spin-up line in the -diagram and demonstrate that such a line cannot be uniquely defined. We derive a simple expression for the amount of accreted mass needed for any given pulsar to achieve its equilibrium spin and apply this to explain the observed differences of the spin distributions of recycled pulsars with different types of companions. From numerical calculations we present further evidence for significant loss of rotational energy in accreting X-ray millisecond pulsars in LMXBs during the Roche-lobe decoupling phase (Tauris 2012) and demonstrate that the same effect is negligible in IMXBs. We examine the recycling of pulsars with CO WD companions via Case BB Roche-lobe overflow (RLO) of naked helium stars in post-common envelope binaries. We find that such pulsars typically accrete of the order of 0.002–0.007 M⊙ which is just about sufficient to explain their observed spin periods. We introduce isochrones of radio millisecond pulsars in the -diagram to follow their spin evolution and discuss their true ages from comparison with observations. Finally, we apply our results of the spin-up process to complement our investigation of the massive pulsar PSR J1614−2230 from Paper I, confirming that this system formed via stable Case A RLO in an IMXB and enabling us to put new constraints on the birth masses of a number of recycled pulsars.