Reduced genetic variation at marker loci in small populations has been well documented, whereas the relationship between quantitative genetic variation and population size has attracted little empirical investigation. Here we demonstrate that both neutral and quantitative genetic variation are reduced in small populations of a fragmented plant metapopulation, and that both drift and selective change are enhanced in small populations. Measures of neutral genetic differentiation (FST) and quantitative genetic differentiation (QST) in two traits were higher among small demes, and QST between small populations exceeded that expected from drift alone. This suggests that fragmented populations experience both enhanced genetic drift and divergent selection on phenotypic traits, and that drift affects variation in both neutral markers and quantitative traits. These results highlight the need to integrate natural selection into conservation genetic theory, and suggests that small populations may represent reservoirs of genetic variation adaptive within a wide range of environments.