Despite increasing information about postglacial recolonization of European freshwater systems, very little is known about pre-Pleistocene history. We used data on the recent distribution and phylogenetic relationships of stone loach mitochondrial lineages to reconstruct the initial colonization pattern of the Danube river system, one of the most important refuges for European freshwater ichthyofauna. Fine-scale phylogeography of the Danubian populations revealed five highly divergent lineages of pre-Pleistocene age and suggested the multiple origin of the Danubian stone loach. The mean sequence divergence among lineages extended from 7.0% to 13.4%, which is the highest intraspecific divergence observed so far within this river system. Based on the phylogeographical patterns, we propose the following hypothesis to relate the evolution and dispersal of the studied species with the evolution of the Danube river system and the Carpathian Mountains: (i) during the warmer period in the Miocene, the areas surrounding the uplifting Alps and Carpathians served as mountainous refuges for cold-water adapted fish and promoted the diversification of its populations, and (ii) from these refuges, colonization of the emerging Danube river system may have taken place following the retreat of the Central Paratethys. Co-existence of highly divergent mtDNA lineages in a single river system shows that range shifts in response to climatic changes during the Quaternary did not cause extensive genetic homogenization in the stone loach populations. However, the wide distribution of some mtDNA lineages indicates that the Pleistocene glaciations promoted the dispersal and mixing of populations through the lowlands.