Communities in isolated habitat patches surrounded by inhospitable matrices often form a nested subset pattern. However, the underlying causal mechanisms and conservation implications of nestedness in regional communities remain controversial. The nested ranks of species in a nested species-by-site matrix may reflect a gradient of species vulnerability to extinction or of colonization ability. However, nestedness analysis has rarely been used to explore determinants of species rank; consequently, little is known of underpinning mechanisms. In this study, we examined nestedness in moorland plant communities widely interspersed within the subalpine zone of northern Japan. Moorland sites differed in area (1000–160 000 m2) and were naturally isolated from one another to various extents within an inhospitable forest matrix. We also determined whether site characteristics (physical and morphometric measures) and species characteristics (niche position and breadth, based on species’ traits) are related to nestedness. Moorland plant communities in the study area were significantly nested. The pH and moorland kernel density (proxy for spatial clustering of moorlands around the focal site) were the most important predictors of moorland site nested rank in a nestedness matrix. Niche breadths of species (measured as variation in leaf mass area and height) predicted species’ nested ranks. Selective environmental tolerances imposed by environmental harshness and selective extinction caused by declines in site carrying capacities probably account for the nested subset pattern in moorland plant communities. The nested rank of species in the nestedness matrix can therefore be translated into the potential order of species loss explainable by species niche breadths (based on variation in functional traits). Complementary understanding of the determinants of site ranking and species ranking in the nestedness matrix provides powerful insight into ecological processes underlying nestedness and into the ways by which communities are assembled or disassembled by such processes.