Understanding the evolution of specialization in host plant use by pollinators is often complicated by variability in the ecological context of specialization. Flowering communities offer their pollinators varying numbers and proportions of floral resources, and the uniformity observed in these floral resources is, to some degree, due to shared ancestry. Here, we find that pollinators visit related plant species more so than expected by chance throughout 29 plant–pollinator networks of varying sizes, with “clade specialization” increasing with community size. As predicted, less versatile pollinators showed more clade specialization overall. We then asked whether this clade specialization varied with the ratio of pollinator species to plant species such that pollinators were changing their behavior when there was increased competition (and presumably a forced narrowing of the realized niche) by examining pollinators that were present in at least three of the networks. Surprisingly, we found little evidence that variation in clade specialization is caused by pollinator species changing their behavior in different community contexts, suggesting that clade specialization is observed when pollinators are either restricted in their floral choices due to morphological constraints or innate preferences. The resulting pollinator sharing between closely related plant species could result in selection for greater pollinator specialization.