Among elephants, the phylogeographic patterns of mitochondrial (mt) and nuclear markers are often incongruent. One hypothesis attributes this to sex differences in dispersal and in the variance of reproductive success. We tested this hypothesis by examining the coalescent dates of genetic markers within elephantid lineages, predicting that lower dispersal and lower variance in reproductive success among females would have increased mtDNA relative to nuclear coalescent dates. We sequenced the mitochondrial genomes of two forest elephants, aligning them to mitogenomes of African savanna and Asian elephants, and of woolly mammoths, including the most divergent mitogenomes within each lineage. Using fossil calibrations, the divergence between African elephant F and S clade mitochondrial genomes (originating in forest and savanna elephant lineages, respectively) was estimated as 5.5 Ma. We estimated that the (African) ancestor of the mammoth and Asian elephant lineages diverged 6.0 Ma, indicating that four elephantid lineages had differentiated in Africa by the Miocene–Pliocene transition, concurrent with drier climates. The coalescent date for forest elephant mtDNAs was c. 2.4 Ma, suggesting that the decrease in tropical forest cover during the Pleistocene isolated distinct African forest elephant lineages. For all elephantid lineages, the ratio of mtDNA to nuclear coalescent dates was much greater than 0.25. This is consistent with the expectation that sex differences in dispersal and in variance of reproductive success would have increased the effective population size of mtDNA relative to nuclear markers in elephantids, contributing to the persistence of incongruent mtDNA phylogeographic patterns.