In this paper we refine the calculation of primordial helium recombination, accounting for several additional effects that were neglected or treated more approximately in previous studies. These include consideration of the following: (i) time-dependent radiative transfer interaction between the 21P1–11S0 and 23P1–11S0 resonances; (ii) time-dependent radiative transfer for the partially overlapping n1P1–11S0, n1D2–11S0 and n3P1–11S0 series with 3 ≤n≤ 10; (iii) electron scattering within a kernel approach. We also briefly discuss the effect of electron scattering and H i quadrupole lines on the recombination of hydrogen. Although the physics of all considered processes is interesting and subtle, for the standard cosmology, with helium abundance Yp≃ 0.24, the overall correction to the ionization history during helium recombination with respect to the previous implementation of CosmoRec remains smaller than |ΔNe/Ne| ≃ 0.05 per cent. The dominant improvement is caused by consistent inclusion of resonance scattering for the 21P1–11S0 resonance. For cosmologies with a large helium fraction, Yp≃ 0.4, the difference reaches |ΔNe/Ne| ≃ 0.22 per cent at z≃ 1800; however, the overall correction to the cosmic microwave background power spectra is small, exceeding |ΔCl/Cl| ≃ 0.05 per cent only at l≳ 3000. In comparison to the current version of Recfast the difference reaches |ΔCl/Cl| ≃ 0.4 per cent at l≃ 3000 for Yp≃ 0.4, and also for the standard value Yp≃ 0.24 we find differences of |ΔCl/Cl| ≳ 0.1 per cent at l≃ 2500. The new processes are now included by the cosmological recombination code CosmoRec and can be activated as needed for most settings without affecting its runtime significantly.