Fragmentation of grasslands and forests is considered a major threat to biodiversity. In the case of plants, the effect of fragmentation or landscape context is still unclear and published results are divergent. One explanation for this divergence is the slow response of long-lived plants, creating an extinction debt. However, this has not been empirically confirmed. In this study, data were compiled from broad-scale studies of grasslands from throughout the world that relate plant diversity to fragmentation effects. Only seven studies from northern Europe, out of a total 61, gave any information on actual habitat fragmentation in time and space. In landscapes with >10% grassland remaining, present-day species richness was related to past landscape or habitat pattern. In landscapes with <10% grassland remaining, in contrast, plant species richness was more related to contemporary landscape or habitat pattern. Studies from landscapes with >10% grassland remaining supported the concept of an extinction debt, while studies from more fragmented landscapes did not provide any evidence of an extinction debt. In order to make generalisations about historical legacies on species diversity in grasslands it is important to consider a range of highly transformed landscapes, and not only landscapes with a high amount of grassland remaining.