Many species have been heavily exploited by man leading to local extirpations, yet few studies have attempted to unravel subsequent recolonization histories. This has led to a significant gap in our knowledge of the long-term effects of exploitation on the amount and structure of contemporary genetic variation, with important implications for conservation. The Antarctic fur seal provides an interesting case in point, having been virtually exterminated in the nineteenth century but subsequently staged a dramatic recovery to recolonize much of its original range. Consequently, we evaluated the hypothesis that South Georgia (SG), where a few million seals currently breed, was the main source of immigrants to other locations including Livingston Island (LI), by genotyping 366 individuals from these two populations at 17 microsatellite loci and sequencing a 263 bp fragment of the mitochondrial hypervariable region 1. Contrary to expectations, we found highly significant genetic differences at both types of marker, with 51% of LI individuals carrying haplotypes that were not observed in 246 animals from SG. Moreover, the youngest of three sequentially founded colonies at LI showed greater similarity to SG at mitochondrial DNA than microsatellites, implying temporal and sex-specific variation in recolonization. Our findings emphasize the importance of relict populations and provide insights into the mechanisms by which severely depleted populations can recover while maintaining surprisingly high levels of genetic diversity.