Plants exhibit tremendous variation in whether mating occurs via self-fertilization versus outcrossing, and many species practice a mixture of both. In theory, the evolution of selfing is influenced by a few large costs and benefits, and the benefits of selfing should be balanced by its costs at equilibrium. We have attempted a cost–benefit analysis of selfing, using population genetic surveys, experimental manipulations and quantitative genetics to a spring-flowering, short-lived perennial, Aquilegia canadensis. This species invests heavily in large, nectar-rich flowers but achieves full seed set in the absence of pollinators by automonous autogamy, and selfs 76% of its seed in natural populations. Floral emasculations reveal that 75% of selfing occurs via autogamy, which increases per-flower seed production (reproductive assurance), but pre-empts ovules and resources that could be used to make outcrossed seed (seed discounting). This is costly because outcrossed progeny appear to survive to maturity 10-fold more often than selfed progeny. Almost all the apparent nonautogamous selfing involves cross-pollination between relatives (biparental inbreeding). This could facilitate the evolution of mixed mating, but not when inbreeding is so strong. Though selfing seems strongly disadvantageous, A. canadensis exhibits striking heritable variation in the separation between anthers and stigmas within flowers (herkogamy), which correlates negatively with selfing. This suggests that the mating system is at equilibrium. There is no applicable theoretical explanation for high selfing in the face of strong inbreeding depression, which occurs in a variety of plants. Understanding the evolution of these enigmatic mating systems remains a major challenge.