The effect of prism adaptation on movement is typically reduced when the movement at test (prisms off) differs on some dimension from the movement at training (prisms on). Some adaptation is latent, however, and only revealed through further testing in which the movement at training is fully reinstated. Applying a nonlinear attractor dynamic model (Frank, Blau, & Turvey, 2009) to available data (Blau, Stephen, Carello, & Turvey, 2009), we provide evidence for a causal link between the latent (or secondary) aftereffect and an additive force term that is known to account for symmetry breaking. The evidence is discussed in respect to the hypothesis that recalibration aftereffects reflect memory principles (encoding specificity, transfer-appropriate processing) oriented to time-translation invariance—when later testing conserves the conditions of earlier training. Forgetting or reduced adaptation effects follow from the loss of this invariance and are reversed by its reinstatement.