Many recent studies on invertebrates have shown how morphology not always captures the true diversity of taxa, with cryptic speciation often being discussed in this context. Here, we show how diversification patterns can be very different in two clades of closely related earthworms in the genus Hormogaster stressing the risk of using nonspecific substitution rate values across taxa. On the one hand, the Hormogaster elisae species complex, endemic to the central Iberian Peninsula, shows morphological stasis. On the other hand, a clade of Hormogaster from the NE Iberian Peninsula shows an enormous morphological variability, with 15 described morphospecies. The H. elisae complex, however, evolves faster genetically, and this could be explained by the harsher environmental conditions to which it is confined—as detected in this study, that is, sandier and slightly poorer soils with lower pH values than those of the other species in the family. These extreme conditions could be at the same time limiting morphological evolution and thus be responsible for the observed morphological stasis in this clade. Contrarily, Hormogaster species from the NE Iberian Peninsula, although still inhabiting harsher milieu than other earthworm groups, have had the opportunity to evolve into a greater morphological disparity. An attempt to delimit species within this group following the recently proposed general mixed Yule-coalescent method showed a higher number of entities than expected under the morphospecies concept, most probably due to the low vagility of these animals, which considerably limits gene flow between distant conspecific populations, but also because of the decoupling between morphological and genetic evolution in the H. elisae complex.