We study the spinning down time-scale of rapidly rotating white dwarfs (WDs) in the frame of the core-degenerate (CD) scenario for type Ia supernovae (SNe Ia). In the CD scenario, the Chandrasekhar or super-Chandrasekhar mass WD is formed at the termination of the common envelope phase or during the planetary nebula phase, from a merger of a WD companion with the hot core of a massive asymptotic giant branch star. In the CD scenario, the rapidly rotating WD is formed shortly after the stellar formation episode, and the delay from stellar formation to explosion is basically determined by the spin-down time of the rapidly rotating merger remnant. We find that gravitational radiation is inefficient in spinning down WDs, while the magnetodipole radiation torque can lead to delay times that are required to explain SNe Ia. To explain the delay-time-distribution of SNe Ia, the merger remnants distribution should be dN/d log (B sin δ) ≈ constant, for 106≲B sin δ≲ 108 G, where B is the dipole magnetic value and δ the angle between the magnetic dipole axis and rotation axis.