Understanding the role of mergers in galaxy formation is one of the most outstanding problems in extragalactic astronomy. While we now have an idea for how the merger fraction evolves at redshifts z < 3, converting this merger fraction into merger rates, and therefore how many mergers an average galaxy undergoes during its history, is still uncertain. The main reason for this is that the inferred number of mergers depends highly upon the time-scale observational methods are sensitive for finding ongoing or past mergers. While there are several theoretical and model-based estimates of merger times, there is currently no empirical measure of this time-scale. We present the first observationally based measurement of merger times utilizing the observed decline in the galaxy major merger fraction at z < 1.2 based on >20 000 galaxies in the Extended Groth Strip Survey and Cosmic Evolution Survey. Using a new methodology described in this Letter, we are able to determine how long a galaxy remains identifiable as a merging system within the CAS system. We find a maximum CAS major merger time-scale of 1.1 ± 0.3 Gyr at z < 1.2, and a most likely CAS merger time-scale of 0.6 ± 0.3 Gyr, in good agreement with results from N-body simulations. Utilizing this time-scale, we are able to measure the number of major mergers galaxies with masses M* > 1010 M⊙ undergo at z < 1.2, with a total number Nm= 0.90+0.44−0.23. We further show that this time-scale is inconsistent with a star formation origin for ultrahigh asymmetries, thereby providing further evidence that structural methods are able to locate mostly merging galaxies.