Gaia, the next astrometric mission of the European Space Agency, will use a camera composed of 106 CCDs to collect multiple observations for one billion stars. The astrometric core solution of Gaia will use the estimated location of the stellar images on the CCDs to derive the astrometric parameters (position, parallax and proper motion) of the stars. The Gaia CCDs will suffer from charge transfer inefficiency (CTI) mainly caused by radiation damage. CTI is expected to significantly degrade the quality of the collected images which ultimately affects the astrometric accuracy of Gaia. This paper is the second and last in a study aiming at characterizing and quantifying the impact of CCD radiation damage on Gaia astrometry. Here we focus on the effect of the image location errors induced by CTI on the astrometric solution. We apply the Gaia Astrometric Global Iterative Solution (AGIS) to simulated Gaia-like observations for 1 million stars including CTI-induced errors as described in the first paper. We show that a magnitude-dependent image location bias is propagated in the astrometric solution, biasing the estimation of the astrometric parameters as well as decreasing its precision. We demonstrate how the Gaia scanning law dictates this propagation and the ultimate sky distribution of the CTI-induced errors. The possibility of using the residuals of the astrometric solution to improve the calibration of the CTI effects is investigated. We also estimate the astrometric errors caused by (faint) disturbing stars preceding the stellar measurements on the CCDs. Finally, we show that, for single stars, the overall astrometric accuracy of Gaia can be preserved to within 10 per cent of the CTI-free case for all magnitudes by appropriate modelling at the image location estimation level and using the solution residuals.