During the last century, unprecedented landscape fragmentation has severely affected many plant species occurring in once widespread semi-natural grasslands in Europe. Fragmentation reduces population size and increases isolation, which can jeopardize the persistence of populations. Recent large-scale ecological and genetic studies across several European countries indicate that fragmented populations of common plant species exhibit a strong genetic differentiation and local adaptation to their home sites, reducing their capacity to establish new populations elsewhere. We discuss the main genetic processes that determine the performance of plant populations in severely fragmented landscapes: namely inbreeding depression, genetic differentiation and genetic introgression. We stress the need for large-scale genetic studies to detect the geographical structure of genetic variation of fragmented plant populations, since nuclei of genetically independent groups of populations may become important targets of conservation. A thorough knowledge on the large-scale geographical structure of genetic variation for a sufficiently wide array of plant species can provide the basis to develop comprehensive conservation plans to preserve the ecological and evolutionary processes that generate and maintain biodiversity of fragmented semi-natural grasslands.