The fundamental effect of the relaxation kinetics of the ensemble of generated valence holes on structure transformations in swift heavy ions tracks in alkali halides is demonstrated. The parameters of the spatial distributions of F-color centers formed in the nanometric vicinities of the trajectories of different ions (from C to U) decelerated in LiF in the electronic stopping regime were extracted from the in situ spectroscopy experiments. Formation of these defects results from self-trapping of valence holes appearing during material ionizations followed by creation and decay of self-trapped excitons. The Monte-Carlo model is applied in order to determine the initial radial distributions of valence holes in tracks of different ions. The detected large difference between the hole and defect distributions indicates fast and long-range diffusion of valence holes from the ion trajectory before their self-trapping. The averaged diffusion coefficient (Dh) of hot holes at the subpicosecond time scale after the projectile passage as well as the conversion efficiencies of valence holes into color centers (β) in relaxing tracks of different ions are estimated (Dh = 0.016–1.4 cm2 s−1, β = 0.01–0.04).