Norway spruce (Picea abies [L.] Karst.) is a broadly distributed European conifer tree whose history has been intensively studied by means of fossil records to infer the location of full-glacial refugia and the main routes of postglacial colonization. Here we use recently compiled fossil pollen data as a template to examine how past demographic events have influenced the species’ modern genetic diversity. Variation was assessed in the mitochondrial nad1 gene containing two minisatellite regions. Among the 369 populations (4876 trees) assayed, 28 mitochondrial variants were identified. The patterns of population subdivision superimposed on interpolated fossil pollen distributions indicate that survival in separate refugia and postglacial colonization has led to significant structuring of genetic variation in the southern range of the species. The populations in the northern range, on the other hand, showed a shallow genetic structure consistent with the fossil pollen data, suggesting that the vast northern range was colonized from a single refugium. Although the genetic diversity decreased away from the putative refugia, there were large differences between different colonization routes. In the Alps, the diversity decreased over short distances, probably as a result of population bottlenecks caused by the presence of competing tree species. In northern Europe, the diversity was maintained across large areas, corroborating fossil pollen data in suggesting that colonization took place at high population densities. The genetic diversity increased north of the Carpathians, probably as a result of admixture of expanding populations from two separate refugia.