Ecology, sexual selection and speciation




Ecology Letters (2011) 14: 591–602


The spectacular diversity in sexually selected traits among animal taxa has inspired the hypothesis that divergent sexual selection can drive speciation. Unfortunately, speciation biologists often consider sexual selection in isolation from natural selection, even though sexually selected traits evolve in an ecological context: both preferences and traits are often subject to natural selection. Conversely, while behavioural ecologists may address ecological effects on sexual communication, they rarely measure the consequences for population divergence. Herein, we review the empirical literature addressing the mechanisms by which natural selection and sexual selection can interact during speciation. We find that convincing evidence for any of these scenarios is thin. However, the available data strongly support various diversifying effects that emerge from interactions between sexual selection and environmental heterogeneity. We suggest that evaluating the evolutionary consequences of these effects requires a better integration of behavioural, ecological and evolutionary research.


The past two decades have yielded empirical support for the hypothesis that divergent selection can drive speciation (Schluter 2009; Sobel et al. 2010). For example, divergent ecological adaptation can reduce the spatial or temporal coincidence of mating, sexual traits and preferences can diverge as a by-product of ecological adaptation, or assortative mating may evolve as a result of selection favouring ecological specialists. At the same time, divergent sexual selection may cause speciation – an insight that Coyne & Orr (2004) considered one of the most important findings in recent speciation research, although the evidence is mostly comparative (Ritchie 2007).

As organisms are subject to both natural and sexual selection simultaneously, important open questions concern how these interact during speciation, and whether such interactions tend to facilitate or rather constrain speciation. Intuitively, one may expect that species characterised by strong sexual selection are more likely to evolve pre-mating isolation. This is because sexual selection directly affects mating patterns and the emergence of linkage disequilibrium (Kirkpatrick & Ravigné 2002), and because sexually selected species often express secondary sexual traits and mating preferences that could be recruited by selection for assortative mating. Moreover, sexually selected traits evolve rapidly and often differ markedly between closely related taxa, and comparative studies indicate that sexually selected taxa may be more species-rich (reviewed in Kraaijeveld et al. 2010).

The null model of sexual selection is the Fisher process (Prum 2010), whereby preferences and traits become genetically correlated, allowing them to coevolve and sometimes ‘run away’ to extreme values. Such coevolution may proceed independently from ecological conditions. However, sexual selection will often be modified or opposed by ecological selection. For example, natural selection may impose a cost on sexual display, causing the rapid coevolution towards more extreme preferences and traits to reach equilibrium where the advantages of the sexual ornament in terms of mating success are exactly matched by disadvantages in terms of survival. Spatial environmental variation in the costs of display can generate between-population variation in equilibrium values of trait and preference. Additional selective forces may interact with Fisherian coevolution. In particular, mate preferences are often subject to natural selection, for example when choosiness entails predation risk or when environmental conditions affect signal perception. Moreover, mate preferences may target display traits that co-vary with ecological performance, such as parasite resistance or foraging ability. These interactions provide several mechanisms by which environmental heterogeneity can affect both mating preferences and sexually selected traits, potentially contributing to population divergence and speciation.

Here, we review the empirical support for the mechanisms by which natural and sexual selection may interact during speciation, and we ask if their interaction tends to facilitate or rather constrain speciation. We focus on intersexual selection in animals and we do not consider sexual conflict, because very little is known about potential interactions with ecology (Fricke et al. 2009).

Theoretical concepts

To facilitate detection of interactions between natural and sexual selection, and to evaluate whether they promote or constrain speciation, we briefly discuss a few definitions and distinctions. Some of these are subject to inconsistent usage and recurrent confusion.

Assortative mating and sexual selection

Assortative mating does not automatically imply sexual selection. Sexual selection emerges from competition and mate choice that generate variation in mating success among individuals of the same sex. If this variation is mediated by heritable traits, sexual selection affects the frequency of trait values and the underlying alleles in a population. In contrast, assortative mating may only reorganise variation by changing genotype frequencies without affecting allele frequencies. Theoretical models and simulation studies of sympatric and parapatric speciation consider a variety of scenarios in this regard. Some analytical models (Kondrashov & Shpak 1998; Kirkpatrick & Ravigné 2002) are explicit about including just assortative mating. In simulation studies this can be more difficult to see. Some apply a normalisation procedure such that all phenotypes obtain equal mating success (e.g. Dieckmann & Doebeli 1999), while others allow selection for mating success in males but not females (e.g. Higashi et al. 1999). These distinctions are not always appreciated, and the terms ‘sexual selection’ and ‘assortative mating’ are often used interchangeably. Herein, we use ‘assortative mating’ when individuals in a mating pair have correlated trait values for one or more ecologically adaptive traits. With ‘sexual selection’ we refer to variation in mating success among individuals of the same sex.

One- and two-allele mechanisms; similarity-based and preference-trait-based mating

Assortative mating between diverging populations, or adaptation-assortative mating within populations, may emerge through the spread of the same allele in both populations or morphs, or of two different alleles (Felsenstein 1981). Examples of the former scenario include alleles that reduce migration, directly increase assortativeness or introduce mating preferences through behavioural imprinting. In a two-allele scenario, different alleles spread in each population, such as alleles for different habitat or mating preferences. In theory, one-allele mechanisms are more conducive to speciation than two-allele mechanisms, particularly when geographical isolation is incomplete, because recombination cannot break the linkage disequilibrium between trait and preference. However, distinguishing them in natural systems can be difficult, as both mechanisms may operate simultaneously or at different stages of divergence (see e.g. Servedio 2008). Moreover, the paradigm is sometimes confused with the distinction between similarity-based mating and mating based on separate preferences and traits.1 In similarity-based mating, assortativeness is mediated by the extent of similarity between individuals with respect to traits that are expressed in both sexes (e.g. body size). Alternatively, non-random mating can be based on a preference (often expressed in females) and a trait (often expressed in males) that are (initially) inherited independently. For instance, some females may prefer large males while other females prefer small males – independently of the females’ own size. The distinction is illustrated in Table 1.

Table 1.   Non-random mating driven by natural selection: mechanisms and hypothetical scenarios
Mechanism of non-random matingLink between natural selection and mate choice
  Pleiotropy: (‘magic’) trait under divergent natural selection also controls non-random matingDirect selection: mate choice divergence favoured by direct selectionIndirect selection: mate choice divergence favoured by selection for offspring fitness
One-allele: the same mating trait spreads in two diverging populationsSimilarityDivergent ecological selection on body size in a population with size-assortative matingSpread of size-assortative mating driven by higher fertilisation rate in size-matched pairs, in a population experiencing divergent selection on body sizeSpread of size-assortative mating driven by selection against intermediates, in a population experiencing divergent selection on body size
Preference-traitPreferences diverge following divergent ecological selection on morphology, because of female imprinting on father’s morphology (independent of own morphology)Sexual selection for ecological performance becomes divergent between alternative ecological regimes – e.g. driven by selection for paternal provisioningSexual selection for ecological performance becomes divergent between alternative ecological regimes – e.g. driven by selection on offspring fitness
Two-allele: two different mating traits spread in two diverging populationsSimilarityImpossible?One subpopulation evolves size-assortative mating because size-matched matings have higher fertilisation rate there; another subpopulation evolves colour-assortative mating because colour-matched matings have lower predation rate there, in a population experiencing divergent ecological selection on colour and sizeOne subpopulation evolves size-assortative mating because of selection against intermediate sizes; another subpopulation evolves colour-assortative mating because of selection against intermediate colours, in a population experiencing divergent ecological selection on colour and size
Preference-traitInfection with habitat-specific parasites changes female sensory abilities, which pleiotropically affect preferences for male ornamentationMale ornaments are habitat-specific indicators of parasite resistance; preference divergence is driven by infection avoidance during matingMale ornaments are habitat-specific indicators of parasite resistance; preference divergence is driven by selection for resistant offspring (reinforcement-like)

Direct and indirect selection on mating traits

This classification concerns the mechanisms by which natural selection affects mating patterns (Table 1). Under direct selection, the genes underlying selective mating also affect survival or fecundity. This happens when certain preferences entail survival benefits (e.g. through reduced parasite transmission), or when ecologically important traits pleiotropically affect mating decisions. For example, environmental conditions may influence the time or place of mating2 or sensory adaptation may affect the perception of sexual signals.

Alternatively, non-random mating evolves in response to indirect selection. Here, preferences become genetically linked to traits that are subject to natural selection. For example, when adaptation-assortative mating preferences spread as a result of selection against maladapted offspring, linkage disequilibrium evolves between the genes underlying adaptation and those underlying mating preference. In secondary sympatry, this mechanism corresponds to one type of reinforcement: the evolution of pre-zygotic isolating barriers, driven by ecological selection against hybrids. When divergence is initiated in sympatry or parapatry, this is sometimes referred to as ‘reinforcement-like’ or ‘analogous to reinforcement’ (e.g. Lorch et al. 2003). Reinforcement mechanisms can be viewed as a special case of sexual selection for ‘good genes’. By increasing the rate and extent of local adaptation, good-genes sexual selection may in theory promote population divergence and speciation even when there is no gene flow to trigger reinforcement (Lorch et al. 2003; Reinhold 2004; Van Doorn et al. 2009).

Speciation is more likely when selective mating is mediated directly by ecologically relevant traits (e.g. body size or habitat choice, but also condition-dependent indicator traits) rather than by arbitrary marker traits (e.g. bristle number on fruit fly legs) (Dieckmann & Doebeli 1999; Kondrashov & Kondrashov 1999; Servedio 2008). Related distinctions are one-trait vs. two-trait (Fry 2003), single-variation vs. double-variation (Rice & Hostert 1993) and magic vs. indicator mechanisms (Servedio 2008): either a single, ‘magic’, trait (Gavrilets 2004) is under divergent natural selection and also determines non-random mating, or non-random mating and ecological adaptation are mediated by two initially independent traits, such that speciation requires the evolution and maintenance of linkage disequilibrium between the two.

While the concept of ‘magic traits’ has become fairly popular recently, it is not always appreciated that the consequences of direct selection or pleiotropy depend on the mechanism of non-random mating. In similarity-based mating, divergent ecological selection may directly generate assortative mating (e.g. divergent selection on host use in insect host races). But in preference-trait mating, including most (if not all) systems with sexual selection, adaptation-assortative mating will always require the build-up of linkage disequilibrium. Direct or pleiotropic effects of ecological selection may concern either preferences or traits, where the former is expected to be more consequential than the latter (e.g. Kirkpatrick & Ravigné 2002). Thus, despite some claims in the empirical literature, divergent natural selection on sexual signals alone does not set the stage for ‘magic trait’ speciation. Signal divergence may lead to assortative mating only in the presence of corresponding preference divergence, which may require strong selection for locally adapted offspring to generate indirect selection on preferences.

Targets and forms of selection

Speciation by natural and sexual selection does not necessarily require that both selective pressures become divergent. For example, when mating success is mediated by condition-dependent ornaments, sexual selection may in theory promote divergent adaptation, even without divergence in preferences or display traits (Lorch et al. 2003; Van Doorn et al. 2009). However, as there is no mechanism in place by which behavioural isolation would evolve, this mechanism is sensitive to migration, particularly by individuals of the more choosy sex. These would accept locally adapted mates in both populations and, depending on their own ecological performance in the new environment, contribute substantially to gene flow. This asymmetry can occur whenever divergent selection does not target mate preferences, and may preclude speciation unless (ecological or sexual) selection against ecological intermediates is sufficiently strong to drive subsequent divergence in preferences and display traits. This implies that divergent selection on signals may be more likely to drive speciation in taxa in which both males and females produce signals and exert mate choice (e.g. mutually ornamented species), or in which the signalling sex disperses more. When divergent selection acts on preferences, conditions for speciation may be less restrictive.

Conversely, divergence may in theory also result from synergistic interactions of stabilising natural selection and disruptive sexual selection. Such a process was investigated in a simulation study by Turner & Burrows (1995): in a population initially fixed for a female preference for conspicuously coloured males, a novel colour preference could spread because predator-mediated selection favoured less conspicuous, intermediate colours. Due to subsequent Fisherian runaway, this new preference and trait could overshoot their ecological optimum. Sexual selection against intermediate signals could then stabilise coexistence of two species residing on opposite sides of the natural selection optimum.

The importance of negative frequency dependence

Strong selection can deplete variation, particularly in the absence of spatial structure. Heritable variation in both ecological and mating traits can be maintained by heterozygote advantage, which will not facilitate speciation. Negative frequency-dependent selection is another possibility. In most theoretical models of ecological speciation with gene flow, negative frequency-dependent selection arises through ecological competition (Gavrilets 2004). Also during sympatric speciation by sexual selection, negative frequency dependence can arise: when intrasexual competition over mating opportunities is more severe among similar than dissimilar phenotypes, individuals with rare ornaments or preferences are at an advantage (Seehausen & Schluter 2004; Van Doorn et al. 2004). Theory suggests that divergence is promoted when both preferences and traits are subject to negative frequency-dependent selection (Van Doorn et al. 2004), implying that examples may be found in systems where both sexes compete over mating opportunities (e.g. mutual mate choice).

Through frequency dependence, natural and sexual selection can act synergistically during divergence: traits that are subject to directional selection along one axis may retain variation as a result of frequency dependence along the other. For example, variation in mate preferences and sexually selected traits can be rescued when trait expression reflects condition (Tomkins et al. 2004), which in turn depends on ecological factors that can generate frequency-dependent selection through resource competition. On the other hand, costs associated with mate searching or assessment may select against rare genotypes, otherwise at a natural selection premium, through Allee effects (e.g. Gavrilets & Boake 1998).

Empirical evidence

Theoretical models, by necessity, are strongly simplified hypothetical representations of nature. Constraints to speciation in theoretical models may have solutions in nature through ecological, genetic or phenotypic dimensions not accounted for in models. Behavioural plasticity is one example: females may not pay the costs of choosiness, in models often prohibitive, if they adjust to mate availability (Kopp & Hermisson 2008). Flexibility of dispersal in response to local conditions is another example (Edelaar et al. 2008). Likewise, theoretical predictions of the relative likelihood of alternative evolutionary mechanisms do not necessarily correspond to their prevalence in nature. For instance, while one-allele mechanisms may be most conducive to speciation in theory, two-allele mechanisms may predominate in speciation in natural systems (Coyne & Orr 2004), and while indirect selection on mating preferences may be a relatively inefficient evolutionary force in theory, if it is common in nature it may contribute substantially to speciation. In the following sections, we consider the empirical evidence for facilitating or constraining effects on speciation arising from interactions between ecological and intersexual selection. We organise our discussion by agents of ecological selection: dietary resources, predators, parasites, sensory environments and other biotic factors. Even though more than one selective agent is implicated in many cases, we think that this categorisation is nevertheless useful because different sources of selection give rise to different interactions with sexual selection. An overview of case studies is provided in Table S1.

Dietary resources

Variation in the availability of alternative dietary resources is the source of divergent or disruptive selection in most theoretical models of ecological speciation. Sexual selection can contribute to diet-mediated speciation by favouring ecological performance. In the North American species complex of crossbills (Loxia curvirostra; Fig. 1, Table S1) up to nine distinct ‘ecotypes’ specialise on extracting seeds from the cones of different conifer species. Despite overlapping distribution ranges, ecotypes are nearly completely reproductively isolated as a result of habitat preference, assortative flocking and assortative mating preferences by females. Ecotype-specific ‘call types’, that are produced by both sexes and learned from the parents, may have evolved as ‘marker traits’ (cf. Kondrashov & Kondrashov 1999) that facilitated the evolution of assortative behaviours (Smith et al. 1999; Snowberg & Benkman 2007). Recent work indicates that female choice exerts directional sexual selection within species as well: females prefer efficient foragers as mates (Snowberg & Benkman 2009). Since foraging efficiency is determined by the interaction of bill morphology and cone type, this preference in principle generates sexual selection for improved adaptation within ecotypes and may simultaneously reduce gene flow between ecotypes. This would essentially be a one-allele mechanism, in which the spread of a single trait, a preference for mates that exhibit efficient feeding, promotes reproductive isolation between divergently adapting populations.

Figure 1.

 Some of the best studied examples for the interaction of sexual selection and ecology in speciation. (a) Red crossbill (Loxia curvirostra): sexual selection for ecological performance becomes divergent, because feeding efficiency is determined by an interaction between bill morphology and conifer cone type. (b) Cricket frog (Acris crepitans): heterogeneity in vegetation has driven call divergence between subspecies. (c) Sockeye salmon (Oncorhynchus nerka): sexual selection for red coloration contributes to population differentiation in carotenoid metabolism between sockeye and kokanee ecotypes. (d) Vidua Indogobirds (photo: V. purpurascens): reproductive isolation through host switches and imprinting on host song. (e) experimental evolution in Drosophila serrata: larval feeding substrate pleiotropically affects cuticular hydrocarbon (CHC) profiles and female CHC preferences. (f) Arctic char (Salvelinus alpinus): sexual selection for parasite resistance may contribute to population differentiation. (g) Strawberry poison frog (Dendrobates pumilio): aposematic coloration mediates both assortative mating and directional sexual selection. (h) Swamp sparrow (Melospiza georgiana): divergent beak morphologies affect male song. (i) Song sparrow (Melospiza melodia): male song and female song preferences covary with vegetation structure. (j) Treehoppers (Enchenopa binotata): females exert sexual selection on male vibrational signals, that are adapted to the transmission properties of different host plants. (k) Darwin’s finches (Geospiza fortis morphs from El Garrapatero): divergent beak morphologies affect male song, potentially mediating assortative mating. (l) Lake Victoria cichlids Pundamilia pundamilia (blue) and Pundamilia nyererei (red): divergence of male coloration may involve sensory drive, sexual selection for ecological performance, and parasite-mediated sexual selection. (m) Passionvine butterflies (Heliconius sp.): sexual selection against hybrid aposematic coloration patterns. (n) Hypoplectrus reef fish (photo: H. puella): mimetic coloration mediates assortative mating. (o) Threespine stickleback (Gasteosteus aculeatus): sexual selection may contribute to divergence through sensory drive, sexual selection for ecological performance, as well as parasite-mediated selection. [Photo credits: (a) A. Wilson; (b) P. Coin; (c) C. Foote; (d) T. Hardaker; (e) A. Morin; (f) I. Folstad; (h) K. Bercaw; (i) L. Blumin; (j) R. Cocroft; (k) A. Hendry; (m) R. Naisbit; (n) L. Ilyes.]

A similar mechanism may occur in anadromous and non-anadromous ecotypes of Pacific sockeye salmon (Oncorhynchus nerka; Fig. 1, Table S1) that have diverged in several life-history traits. On shared spawning grounds, size-assortative mating maintains reproductive isolation between morphs, while sexual selection on coloration contributes to adaptive differentiation. Differences in carotenoid availability between marine (high) and lacustrine (low) feeding habitats have led to heritable differences in the efficiency of carotenoid metabolism (Craig & Foote 2001). Sexual selection favours bright red coloration in both ecotypes and thereby promotes phenotypic convergence. At the same time, sexual selection promotes adaptive divergence, because it selects for increased efficiency of carotenoid uptake in the freshwater residents, and reduces the mating success of hybridogenic lake residents that will have weak colour expression (Craig et al. 2005). This may be one of the few documented cases in which directional sexual selection for indicators of local adaptation might contribute to population differentiation, without mate preferences or indicator signals diverging themselves (Van Doorn et al. 2009). It also illustrates, however, how migration affects divergence: hybrids that migrate to the carotenoid-rich marine habitat should achieve brighter red coloration and consequently higher mating success than non-hybrids, contributing to gene flow between ecotypes. If sexual selection exclusively targets carotenoid metabolism, without promoting differentiation in other ecologically important traits, its effects on hybrid fitness and therefore speciation may be limited.

When the traits that mediate resource exploitation are also used in sexual signalling, divergence in resource use can change sexual signals. In several bird species, divergent morphological adaptation to alternative diets coincides with changes in song characteristics. Bill morphology has therefore been proposed to be a ‘magic trait’: it is subject to divergent natural selection and also mediates adaptation-assortative mating. However, when song characteristics are subject to sexual selection as well, this may either promote or constrain speciation. An example occurs in North American swamp sparrows (Melospiza georgiana; Fig. 1, Table S1) that have recently colonised coastal marshes and evolved larger beaks to enable feeding on aquatic invertebrates. Divergence in beak size affects mating success: smaller beaks facilitate the production of high-performance songs (Ballentine et al. 2004), that are preferred by females (Ballentine 2006). Sexual selection may hence hamper adaptive divergence in this system. However, coastal males appear to compensate by producing more complex songs than males in inland populations (Ballentine 2006), implying that female preferences for song complexity could promote reproductive isolation. Testing this scenario requires comparing female preference functions for song traits between inland and coastal populations of swamp sparrows.

Also in Darwin’s finches, archetypes of adaptive radiation and the inspiration for the idea that morphology affects song and thereby mating patterns (Podos 2001), mechanisms of preference divergence are unresolved. Some recent work focused on a population of Geospiza fortis (Fig. 1; Table S1) in which variation in beak size is bimodally distributed, perhaps providing opportunities to investigate the early stages of divergence. Different beak sizes produce different bite forces and different song (Table S1). Mating is largely assortative by beak size (Huber et al. 2007), but several mechanisms may account for this. First, it is not clear whether the phenotypes involved are really G. fortis beak size morphs or the result of hybridisation with Geospiza magnirostris (Grant & Grant 2010). Second, it is unknown whether assortative mating is mediated by song or by bill morphology itself – or even other traits. Third, preferences may be learnt through imprinting, which would constitute a one-allele, magic trait speciation mechanism, or they may diverge through indirect selection because natural selection against intermediates may favour assortative mating (Grant & Grant 2008). Both mechanisms may or may not involve sexual selection.

Another example of diet-mediated signal divergence is the cuticular hydrocarbons (CHCs) that mediate mate recognition in many insects (Smadja & Butlin 2008). In cactophilic Drosophila mojavensis, different feeding substrates have direct effects on CHC composition, and can result in partial mating isolation (Table S1). Experimental evolution in laboratory populations of Drosophila serrata (Fig. 1) show that substrate-mediated divergence of CHC profiles and CHC preferences can affect male mating success (Rundle et al. 2009). It seems intuitive that CHC profiles could signal local adaptation, generating indirect divergent selection on female mating preferences between alternative environments. However, the D. serrata experiment did not support this: sexual selection did not promote adaptation to novel substrates (Rundle et al. 2006), leaving the mechanism by which natural and sexual selection interact in this system unresolved. Possibly, sexual selection for the male trait that signalled adaptation in the ancestral habitat initially works against adaptation to novel environments (similar to the coastal swamp sparrows).

Dietary differentiation appears to play a major role in divergence of three-spine stickleback (Gasterosteus aculeatus species complex; Fig. 1, Table S1) in coastal lakes in British Columbia: sympatric limnetic and benthic species exploit alternative habitats and differ in a number of morphological traits that likely contribute to foraging efficiency and predator avoidance in alternative habitats. At least one of these traits is important in mate choice: benthics and limnetics differ in body size, and mate choice studies have revealed size-assortative mating (Table S1). A ‘magic trait’ mechanism has been suggested, where isolation between ecotypes would have evolved as a by-product of adaptation in body size to divergent habitats. This scenario assumes pre-existing preferences for similarly sized mates that translate divergent natural selection on body size to adaptation-assortative mating. However, recent work suggests that neither males nor females exert size-related mating preferences within either benthics or limnetics, nor in their presumed anadromous ancestor (Head et al. 2009). This result is surprising because Rundle & Schluter (1998) found evidence for reinforcement and sexual selection against intermediates in the same species pair (Paxton Lake). Possibly, sexual selection and assortative mating are mediated by other traits, such as body shape or male coloration, that correlate with body size between, but not within species (see below).


Variation in predation regimes can drive population differentiation in traits that confer survival advantages, such as locomotor performance, habitat choice, life history traits and morphology. When such traits are involved in mate selection, their divergence may cause reproductive isolation as a by-product. This mechanism may proceed without selection on mating success, but there are various ways in which sexual selection may interact with predator-mediated selection: predators impose mortality costs on sexual ornaments and mate assessment, and predation avoidance itself can be a target of mate choice. These mechanisms provide considerable opportunity for diversifying interactions between natural and sexual selection. The idea that these interactions could contribute to speciation has been around for a while (Verrell 1991), but empirical evidence remains scarce.

In addition to the difficulty of studying predation in the wild, there may be a more fundamental reason for this paucity of data. Predation risk increases the cost of mate assessment, which theoretically reduces speciation probability when gene flow is non-zero. In allopatry, however, differences in the costs of choosiness or ornamentation can, through Fisherian coevolution between the sexes, generate divergence in preferences and ornaments. Perhaps, if interactions between sexual selection and predation play a role in speciation, we should look for examples in taxa that experience different predation regimes in allopatry.

Decades of research in the Trinidadian guppy (Poecilia reticulata) have provided some of the best evidence for allopatric predator-induced variation in sexual communication. In upstream river stretches with low predation regimes, male coloration is more elaborate and conspicuous than in downstream habitats where predators abound (Houde & Endler 1990; Schwartz & Hendry 2007). Correspondingly, female preferences for colourful males are weaker, absent or reversed in high-predation populations (Houde & Endler 1990; Schwartz & Hendry 2007). But despite the evidence for rapid co-adaptation of male and female reproductive strategies, the Trinidadian guppies do not appear to have speciated. Pre-mating isolation is weak, geneflow is extensive and population differentiation is associated with spatial isolation rather than divergent predation regimes (Crispo et al. 2006). Possible explanations include high levels of migration (particularly downstream) and male mating strategies that circumvent female choice (Magurran 1998).

Predation could also promote population differentiation by reducing migration. This may occur in three-spine sticklebacks in lakes in Iceland, that adapt to different microhabitats with either lava or muddy and vegetated substrates (Olafsdottir et al. 2006). Differences between morphs in body armour indicate also a more direct effect of predation regime on differentiation. Sexual selection is implicated by the observation that lava males build less structured nests, which makes them less attractive for females of the other morph. However, a recent analysis of genetic population structure suggests that eco-morphological differentiation may not be associated with substantial reproductive isolation (Olafsdottir & Snorrason 2009).

Aposematic signals, that warn predators about prey unprofitability, are anti-predator adaptations that could become targets of sexual selection in a ‘good genes’ like mechanism. In taxa as diverse as butterflies and poison frogs, aposematic signals are not only used for predator warning but also in mate selection. It is tempting to speculate that the dual role of aposematic signals in these taxa has contributed to their enormous colour and species diversity, and some have suggested that warning colours may function as ‘magic traits’ (e.g. Gavrilets 2004). In several species of Heliconius butterflies, genes for colour are tightly linked to genes for colour preference (Fig. 1; Table S1; Kronforst et al. 2006; Melo et al. 2009). This may help to maintain once evolved preference-trait associations in the face of secondary gene-flow, and facilitate hybrid speciation, but it may not explain how these associations evolved in the first place. In contrast, colour and preference appear to evolve independently in the polymorphic poison frog Dendrobates pumilio (Fig. 1, Table S1; Maan & Cummings 2008). Whether or not warning-colour-assortative preferences result in sexual selection is often unclear (see Table S1). In both Dendrobatid poison frogs and Heliconius butterflies, the opportunity for sexual selection differs markedly between species, as a result of variation in mating systems and parental investment (Summers et al. 1997; Cardoso et al. 2009). This variation provides opportunities for future studies to clarify the role of sexual selection in aposematic divergence and speciation.


Parasite-mediated sexual selection may be one of the best studied forms of interaction between ecology and sexual selection. Several properties of parasites suggest that these interactions may facilitate population divergence. First, just like predators, parasites may exploit sexual signals to locate their hosts and thereby exert selection on signal design. Second, parasite infestation has direct effects on host condition, influencing mate attraction and selection. Third, parasite exposure is closely linked to environmental conditions, in particular to habitat and dietary resources. Fourth, rapid adaptation of parasites to host defences initiates co-evolutionary cycles that help to maintain genetic variation and ensure honesty in sexual signalling.

Parasites may directly influence mate choice, for example, when infection reduces the energy available for mate assessment and thereby weakens sexual selection. Divergence would be more strongly promoted when parasites change the target or the direction of sexual selection. Novel preferences in infected populations could initiate divergent coevolution of preferences and traits. Current evidence for such effects is scant. Most studies of parasite-mediated sexual selection concern the indirect effects of parasites on mating preferences: the trade-off between parasite resistance and sexual ornamentation allows choosy individuals to select more healthy partners that may confer heritable resistance to the offspring. This mechanism may contribute to speciation: when populations diverge in habitat or diet, and thereby parasite exposure, sexual selection for parasite resistance may accelerate adaptation and promote reproductive isolation. Supporting evidence comes mostly from freshwater fish but is purely correlational. For example, correlated divergence in diet, habitat, parasite load and sexually selected coloration has been found in Arctic char (Skarstein et al. 2005), cichlids (Maan et al. 2008) and sticklebacks (MacColl 2009) (Fig. 1; Table S1). To address the underlying causal relationships, studies should test whether species-specific female mating preferences enhance offspring resistance to species-specific parasites.

Rather than acting through indicator traits, sexual selection may target parasite resistance genes directly. In vertebrates, the major histocompatibility complex (MHC) could be subject to divergent sexual selection. This highly polymorphic gene family encodes glycoproteins that recognise pathogen-derived peptides. Mating preferences for MHC genotypes have been demonstrated in several taxa, typically mediated by the olfactory signatures of MHC gene products. In the face of highly diverse infection challenges, selection not only acts on individual alleles but also on within-individual MHC allele diversity. Individual fitness is believed to be optimised by intermediate MHC diversity: low diversity allows some parasites to go unrecognised, but very high diversity entails the risk of autoimmune reaction (Woelfing et al. 2009). Balancing selection contributes to the maintenance of genetic variation within populations, an important prerequisite for speciation. Optimising allelic diversity in the offspring entails that MHC-mediated mate choice depends on the MHC genotype of both partners, implying that divergent natural selection on MHC genotypes may affect both preferences and traits pleiotropically. This inspired the hypothesis that MHC-based immune defence could mediate ecological speciation (Blais et al. 2007; Nuismer et al. 2008; Eizaguirre et al. 2009a).

In northern Germany, riverine populations of three-spine stickleback are infected by fewer parasite species and show lower MHC diversity than lacustrine populations (Table S1). There is also experimental evidence for habitat-specific parasite resistance (Scharsack et al. 2007). Within lake populations, mating success is affected by MHC allelic diversity as well as by particular MHC alleles (see Eizaguirre et al. 2009b and references therein). Sexual selection may thus accelerate divergent adaptation and contribute to reproductive isolation, either through indirect selection favouring female choice for specific alleles, or through pleiotropic effects of MHC divergence on preferences. Currently, evidence for either mechanism is lacking, because mate preferences have not been compared between river and lake populations. There seems to be little scope for reinforcement, as hybrids with intermediate MHC genotypes did not suffer increased infection in reciprocal transplant experiments (Rauch et al. 2006).

Rapid divergence in MHC genes and MHC-mediated mate choice have been documented in several vertebrate taxa, but so have the maintenance of ancient MHC polymorphisms across species boundaries (Piertney & Oliver 2006). Investigations in the context of speciation have only just begun, and will need to be intensified to understand the role of MHC evolution in species divergence.

Sensory environment

Animal communication is influenced by the sensory environment in which it takes place: the medium through which signals are propagated and the background against which they are perceived. Variation in these parameters may exert diversifying selection on both senders and receivers (Endler 1992). When signals mediate selective mating, heterogeneous signalling environments could promote divergence. Reproductive success may be maximised by different signal designs in different habitats. There are many empirical examples of environmental effects on signal conspicuousness, mostly from acoustic and visual signals. However, when signal perception is a direct consequence of signal design and habitat-specific transmission, without divergence in sensory systems or mating preferences, speciation may require additional mechanisms (Schluter & Price 1993; Chunco et al. 2007), such as geographical barriers, active habitat choice or habitat-specific condition-dependent signal expression. Divergence may be more likely when sensory environments exert direct or indirect selection on mating preferences.

The same environmental conditions that affect signal design can also exert selection on sensory properties. Divergent natural selection for sensory performance in different environments, or for detecting different targets in the same environment, will generate sensory biases that may pleiotropically affect mating preferences. Alternatively, locally adapted signals can become associated with other traits that determine local ecological performance (including sensory systems), leading to indirect selection for divergence in preferences.

The strongest evidence for divergent sensory drive in visual communication comes from fish (Boughman 2001; Seehausen et al. 2008). This is probably because water is an optically dense medium, in which different colours of light penetrate to very different extents. Together with variation in the kind and density of suspended and dissolved matter, this generates steep environmental gradients. There are two model systems in which divergent sensory drive is strongly implicated in speciation: North-American three-spine sticklebacks and East-African haplochromine cichlids (Fig. 1).

In addition to body size and armour, trophic morphology and parasite resistance, allopatric populations and sympatric species of three-spine sticklebacks diverge in male nuptial coloration. In North-American waters with red-shifted ambient light, the archetypical bright red throats of breeding males are replaced to variable degree by a darker, sometimes solid black coloration (Reimchen 1989). Boughman (2001) showed that this variation correlates with female visual sensitivity to red light and with the strength of female preferences for male red coloration. Moreover, the extent of divergence among populations in signals and preferences is correlated with the extent of assortative mating among them in the laboratory, suggesting that divergent sexual selection generated by sensory drive may contribute to speciation (Table S1).

Similar correlations are observed in a pair of closely related cichlid species from Lake Victoria. Pundamilia pundamilia and Pundamilia nyererei (Fig. 1, Table S1; Seehausen et al. 2008) have diverged in male nuptial coloration (blue vs. red) and associated female mating preferences. While they are geographically sympatric at islands in Lake Victoria, they inhabit different light environments. P. pundamilia occupies shallow waters with broad-spectrum daylight. P. nyererei tends to occupy deeper waters with darker and more yellowish light conditions. Female preferences for conspicuous male colours could lead to divergent sexual selection because conspicuousness is habitat-dependent: in deeper waters, red colours reflect more light than blue colours. However, in addition, the two species carry different visual pigment alleles. The extent of divergent adaptation in the visual system and the extent of genetic differentiation at neutral marker loci both vary with the extent of depth segregation, such that gene flow between red and blue is more common where the difference in depth and in visual system is smaller. Together, this suggests that divergent sensory drive is involved in Pundamilia speciation (Maan & Seehausen 2010).

For both the cichlids and the sticklebacks, however, the underlying mechanism is unresolved: are female preferences for male colours pleiotropic by-products of visual adaptation to particular light environments? Or are visual sensitivity, female preferences and male coloration independent traits that become associated due to reinforcement or perhaps Fisherian coevolution? Distinguishing these alternatives requires experimental manipulation of visual sensitivity (not female perception or male colour, as has been done) and subsequent analysis of female preferences.

Also in terrestrial habitats, correlations between spectral environments and visual signal design have been documented in birds, reptiles and insects. However, evidence for associated variation in visual sensitivity is limited (but see Leal & Fleishman 2002) and little is known about potential consequences for (divergent) sexual selection and speciation.

In addition to visual signals, terrestrial vegetation can have marked effects on the transmission of acoustic and vibrational signals. Birds tend to produce slower and lower-frequency songs in more densely vegetated habitats (Slabbekoorn & Smith 2002; Price 2008), and even change their song in urban environments (Slabbekoorn & Ripmeester 2008). Few studies explore associated variation in acoustic perception and mate choice. Differences in both male song and female preferences have been documented between subspecies of song sparrows (Melospiza melodia; Fig. 1) that overlap in a hybrid zone in southern California and inhabit distinct signalling environments (Patten et al. 2004). These differences likely contribute to reproductive isolation in the hybrid zone, but the mechanism underlying preference divergence has not been identified. Males respond more aggressively to songs from their own population, but this difference in response was found to decrease in the hybrid zone. This indicates the absence of negative frequency-dependent selection that would facilitate coexistence in sympatry, and perhaps the failure of reinforcement.

Correspondence between acoustic sensitivity, preferences and signals has been demonstrated for several species of frogs and may contribute to species divergence and coexistence. However, the role of ecology in driving these correlations is often unknown, and the consequences for speciation remain unclear. In cricket frogs (Acris crepitans; Fig. 1), variation in body size pleiotropically affects both male calls and female auditory tuning, but these effects do not fully explain population divergence (Ryan et al. 1992). Divergence in calls and preferences is additionally shaped by environmental heterogeneity in acoustic transmission and noise levels (Ryan et al. 1990; Witte et al. 2005). The effect of this variation on genetic differentiation remains to be quantified. More generally, it appears that environmental heterogeneity in acoustic transmission is of limited importance in frog speciation (Kime et al. 2000).

Habitat-dependent divergence in vibrational signals appears to play an important role in the diversification of Enchenopa binotata treehoppers, a North-American complex of 11 closely related sap-feeding insect species that specialise on different host plants (Fig. 1; Table S1; Cocroft et al. 2008). Limited dispersal, together with differences in host plant phenology, cause substantial isolation in time and space. Yet, individuals of different species do meet and interact, suggesting that behavioural isolation may be important. For mate attraction, male E. binotata produce vibrational signals that travel through the stems and petioles of their host plants. Host plant shifts coincide with shifts in signalling frequency, such that male signals optimally exploit the transmission properties of the plant (McNett & Cocroft 2008). This implies that sexual selection for more effective signals has contributed to male signal divergence. However, male mating success is not simply a consequence of signalling efficiency: females exhibit species-specific preferences for several signal characteristics (Rodriguez et al. 2006) that cause strong sexual selection within populations (Sullivan-Beckers & Cocroft 2010). Some aspects of sexual communication have thus diverged independently from host plant transmission properties, and the contribution of sensory drive to the initiation of speciation remains to be evaluated. Preferences may have diverged under indirect selection: larval development and survival are host plant-specific, such that male signal characteristics could mediate a reinforcement-like process (Table S1). Testing this hypothesis requires measurement of hybrid fitness.

Other abiotic factors

(Micro-) climatic variation exerts diversifying selection on various traits, which may affect selective mating. For example, in many insects, wing melanisation functions in thermoregulation as well as in species and mate recognition. A few studies have explored how these two functions interact. In Colias butterflies, natural selection favours increased wing melanisation at higher elevations, but males prefer less-melanised females across the elevational range – inhibiting adaptive divergence (Ellers & Boggs 2003).

Insect CHCs are another example of traits that mediate both climatic adaptation (e.g. resistance to cold and drought) and mate choice. As a result, climate-driven divergence in CHC quality or quantity may pleiotropically cause behavioural isolation. In Drosophila, polymorphic desaturase genes are associated with variation in both CHC composition and mate discrimination (e.g. Marcillac et al. 2005), implying that a relatively simple genetic change could facilitate speciation. However, while it is clear that the desaturase genotype affects CHC, it is unclear whether or not this is enough to change mating patterns. Hence, the mechanistic link between desaturase genotype, climatic adaptation and sexual isolation remains to be established (Coyne & Elwyn 2006; Greenberg et al. 2006). Moreover, sexual dimorphism in CHC’s is common, and male and female CHC’s can evolve independently (e.g. Higgie & Blows 2008). This indicates that both natural and sexual selection may contribute to CHC divergence, independent of pleiotropy.

Climatic variation may influence animal life history traits and mating systems, thereby potentially affecting the strength and direction of sexual selection. In birds, harsh conditions at higher elevations favour greater biparental care, leading to weaker sexual selection on males (Badyaev & Ghalambor 2001). Accordingly, species of cardueline finches that breed at higher altitudes produce less elaborate songs than those that breed at lower altitudes (Snell-Rood & Badyaev 2008). However, currently there is no evidence that speciation happened by divergent selection along altitudinal gradients in these birds. Climatic factors may influence not only the strength but also the targets of sexual selection. Based on a correlation between climate variability and song elaboration in mockingbirds, Botero et al. (2009) suggested that sexual selection may favour more complex signals in unpredictable climates, because they could serve as reliable indicators of learning ability. Testing these ideas, and determining possible contributions of climate-driven signal variation to speciation, requires analysis of preference and trait evolution along climatic gradients.


Both theory and data lend strong support for the prevalence of interactions between sexual and ecological selection. However, based on the studies reviewed here, we have to conclude that we still know very little about how these interactions may contribute to speciation in nature. This may be surprising to some. However, investigating whether and how interactions between sexual and ecological selection facilitate or constrain speciation requires integrative research. Presently, we found that while there may be fairly strong support for some components of possible speciation-facilitating mechanisms, only a few studies (or collections of studies) provide convincing evidence for all the necessary ingredients to a hypothesised speciation scenario.

As one deficiency that stands out in particular, we encountered many empirical model systems for speciation in which the involvement of sexual selection is highly likely, but never explicitly addressed. For example, the joint occurrence of acoustic imprinting and ecologically mediated song divergence may be very important in bird speciation (Price 2008), but in many cases it is unclear whether the song characteristics that mediate reproductive isolation are subject to sexual selection within species. Such correspondence cannot simply be assumed, because interspecific and intraspecific mate choice are often mediated by different signals (Ptacek 2000).

To some extent, the empirical disconnect between assortative mating and sexual selection may be due to a disciplinary divide, where speciation biologists study the former and behavioural ecologists the latter. Ecological speciation research for instance is strongly focused on the ecological causes and consequences of assortative mating, often disregarding the possibility that sexual selection is involved as well, either inhibiting or accelerating divergent adaptation. Conversely, studies of geographical variation in sexual communication often fail to measure the consequences for gene flow. Variation in sexual signals is more easily and much more frequently documented than variation in preferences, but to make inferences about speciation, measuring variation in preferences is at least as important as measuring variation in signals.

Theory predicts that direct selection on mating preferences should be more consequential than indirect selection. At the same time, the concept of ‘magic traits’ has received considerable attention in the empirical speciation literature recently. The evidence reviewed here, however, did not provide a single conclusive example of speciation by direct selection or pleiotropic effects on mating preferences. To determine whether or not this reflects an actual prevalence of indirect over direct mechanisms in nature, experimental studies are required that allow distinguishing between the two.

As noted above, there is no lack of evidence for direct effects of ecological selection on sexual communication systems. Due to paucity of data on mating preferences, migration rates and gene flow, however, the consequences for speciation remain unclear in almost every case. Moreover, many studies report effects on choosiness or sexual motivation, rather than changes in preference functions (but see e.g. Holveck & Riebel 2009). Variation in the strength of sexual selection may not be particularly powerful in driving divergence, when the target and direction of sexual selection remain unchanged. This is because its effect on reproductive isolation is asymmetrical. For example, male finches with elaborate song that migrate to higher altitudes may suffer increased mortality, but not necessarily a reduction in mating success.

Theoretically, the common phenomenon that sexually selected signals co-vary spatially with environmental variables provides a mechanism by which ecological adaptation can be exposed to sexual selection, which could accelerate adaptive population divergence. However, there is only limited evidence that such ‘good-genes’ or ‘indicator’ mechanisms lead to assortative mating and speciation. In fact, we saw several cases in which sexual selection may rather inhibit ecological adaptation (e.g. in swamp sparrows, Drosophila and Colias), because ancestral preferences exert selection in directions opposite to those favoured by natural selection in a novel environment. It will be important to establish whether this is a common phenomenon, whether sexual selection in such instances is actually maladaptive, and to identify the factors that determine whether and how this potentially transient state can be overcome.

Ecological speciation may often be characterised by the operation of several mechanisms simultaneously or in close succession. This is no coincidence, but a consequence of the multidimensional nature of ecology. In fact, speciation may be more likely when several mechanisms interact (Nosil et al. 2009). It seems plausible, for example, that a sensory biased mate preference spreads more rapidly when it contributes to offspring parasite resistance. This complexity presents challenges for reconstructing the sequence and relative importance of events in speciation.

Several major questions remain to be addressed. For example, are there general patterns regarding the effects of different agents of ecological selection on mating preferences, such that some may initiate preference divergence more often through direct selection or pleiotropy (e.g. variation in signalling environments, parasites), while others are more likely to act through indirect selection (e.g. dietary resources, predators)? Are there patterns regarding the traits that predict speciation propensity through interactions between sexual and ecological selection, such as variation in mating systems, life history, the strength of sexual selection and the importance of indicator traits and direct benefits? Does sexual selection merely accelerate a process that is initiated by ecological selection, or can it be a prerequisite for ecological speciation? Is there speciation by sexual selection alone? And finally, what is the role of ecology in shaping intersexual and intrasexual conflict, and could these interactions contribute to speciation?

To answer these and other questions, research should (1) address the ecological relevance of variation in mating traits and the mechanisms underlying population divergence in these traits; (2) address the relevance to sexual selection of variation in ecological traits and population divergence in these traits; (3) investigate interspecific mate choice and intraspecific sexual selection in the same experimental and conceptual context; and (4) relate variation and divergence in mating traits and preferences to gene flow and genetic population differentiation. There is no doubt that ecological heterogeneity influences sexual selection and mate choice, but evaluating the macro-evolutionary consequences of these effects requires a better integration of behavioural, ecological and evolutionary research. Further, it is often suggested that speciation mechanisms are best reconstructed by studies of very recent or ongoing population divergence. But to understand the relative importance of natural and sexual selection, their interactions and the sequential involvement of different mechanisms, researchers should take advantage of natural variation in the progress to speciation (Hendry 2009; Seehausen 2009). We therefore suggest that integrative studies of populations at different levels of reproductive isolation are needed to evaluate how sexual and ecological selections interact during speciation in nature.


  • 1

    Preference-trait assortment has also been called ‘split-trait’ (Kondrashov et al. 1998) or ‘matching-based’ mating (Gavrilets 2004).

  • 2

    Coyne & Orr (2004) use the term ‘no-gene mating traits’ when reproductive isolation is an immediate by-product of ecological divergence, without additional trait evolution.


We thank Katie Wagner, Carole Smadja, Michael Ryan, Ken Kraaijeveld and three anonymous referees for comments and discussion. Financial support was provided by the Swiss National Science Foundation to both authors.