Because of maternal mtDNA inheritance, mtDNA mutations detrimental only in males are not expected to be selected against, an effect termed the “mother's curse.” However, if there is positive-assortative mating, equivalent to what was called “inbreeding” by Wade and Brandvain (2009), then selection can act to reduce the frequency of these male-specific detrimental mtDNA mutants. On the other hand, as shown here negative-assortative mating, or “outbreeding, ” paradoxically can result in an increase in the frequency of male-specific detrimental mtDNA mutants. The implications of these findings are briefly discussed.

Because mtDNA is maternally inherited, theory predicts that mutations that have large detrimental effects in males but only mild or no detrimental effects in females may exist in substantial frequencies (Frank and Hurst 1996). This sex-dependent difference in selection has been termed the “mother's curse” (Gemmell et al. 2004) to emphasize the inability of selection to act on maternally inherited mtDNA mutants that primarily have detrimental effects in their males. Evidence supporting this effect has been found in beetles (Dowling et al. 2007), Drosophila (Innocenti et al. 2011; Rand et al. 2001), chickens (Froman and Kirby 2005), hares (Smith et al. 2010), and humans (Ruiz-Pesini et al. 2004; Holyoake et al. 2007).

Recently, Wade and Brandvain (2009) suggested that the negative impact of such mtDNA mutants in males can be selected against given that there is either inbreeding (see also Unckless and Herren 2009) or kin selection. Wade and Brandvain (2009) concluded that “transmission mode does not dictate a gene's evolutionary destiny because there are many ways in addition to transmission to achieve a positive regression between a gene and its fitness effects.” However, there is no inbreeding for haploid genes, defined as identity-by-descent, and they defined inbreeding as “the relatedness between a male and his mate.” In fact, what they examined is identical to a model of positive-assortative mating. Therefore, below I first present a model of positive-assortative mating and then examine the impact of negative-assortative mating, a surrogate of “outbreeding. ”

Model

Let us assume that there are two mtDNA variants, A_{1}, the normal wild-type variant, and A_{2}, a new mutant with a detrimental effect in males and no detrimental effect in females with frequencies p_{1} and p_{2}, respectively. Given random mating, there are the four possible mating types in the population as given in Table 1. Assume that a proportion R of the population mates at random, a proportion P is from positive-assortative mating, and a proportion N is from negative assortative mating (R+P+N= 1).

Table 1. The four different mating types and their frequencies when there are mtDNA variants wild-type A_{1} and mutant A_{2}, and there are proportions R of random mating, P of positive-assortative mating, and N of negative-assortative mating when there is polygamy and monogamy (given that the frequency of the mutant is below or above 0.5). Also given are the relative fitness for selection s_{m} against families with A_{2} fathers.

Mating type

Male

Female

R

P

N

Selection in A_{2} father families

Polygamous

Monogamous

p_{2}≤0.5

p_{2}≥0.5

M_{11}

A_{1}

A_{1}

p_{1}

–

1−2p_{2}

–

1

M_{12}

A_{1}

A_{2}

p_{2}

p_{2}

p_{1}

1

M_{21}

A_{2}

A_{1}

–

p_{1}

p_{2}

p_{1}

1−s_{m}

M_{22}

A_{2}

A_{2}

p_{2}

1−2p_{1}

1−s_{m}

In the proportion P of the population that is from positive-assortative mating, there are matings A_{1}×A_{1} and A_{2}×A_{2} with the same mtDNA type with frequencies p_{1} and p_{2}. The proportion N of the population that is from negative-assortative mating are those that have different mtDNA types or A_{1}×A_{2} and A_{2}×A_{1}, where the first individual is the male. If we assume polygamy, that is, males can mate with multiple females and that all females are mated by the males present, then the frequency of these mating types are determined by the frequency of the females. Therefore, the frequency of mating type A_{1}×A_{2} is p_{2} and the frequency of mating type A_{2}×A_{1} is p_{1}.

When there is monogamy and negative-assortative mating, then the frequency of the four mating types is determined by assuming that mating occurs as outlined in Table 2. First, as many monogamous pairs as possible, given the frequency A_{2}, are generated. Then the remainder of the population mates monogamously but these matings are all A_{1}×A_{1} because these are the only individuals left. The total frequencies of the four mating types are the sum of these two groups and are given in Table 2(c).

Table 2. Determination of negative-assortative mating frequencies when mating pairs are monogamous, assuming that p_{1} > p_{2}. First, (A) the proportion of pairs that mate in a negative-assortative manner are determined, then (B) the frequency of the mating pairs of the remaining individuals determined. The sum of these then give (C), the overall frequency of mating pairs. The same logic can be used to determine the mating type frequencies when p_{1} < p_{2}.

A_{1} (p_{1})

A_{2} (p_{2})

(A) Negative-assortative mating

A_{1} (p_{1})

p_{2}

A_{2} (p_{2})

p_{2}

(B) Mating of remainder

A_{1} (p_{1})

p_{1}−p_{2}

A_{2} (p_{2})

(C) Total

A_{1} (p_{1})

1−2p_{2}

p_{2}

A_{2} (p_{2})

p_{2}

Using the frequencies of these four mating types, and the proportions of R random, P positive-assortative, and N negative-assortative mating (with polygamy) are as given in Table 1, then the frequencies of the four mating types become

(1)

Now assume that matings where the male is A_{2} have a lower relative number of progeny, 1 −s_{m}, compared to matings where the male is A_{1}. This occurs because the mutant A_{2} in males impairs sperm function or male fertility (Gemmell et al. 2004; Wade and Brandvain 2009). Therefore, the frequencies of the mating types become

(2)

where

Because these variants are maternally inherited, the frequency of A_{2} in the progeny generation is the frequency in females or

(3)

Results

POSITIVE-ASSORTATIVE MATING AND SELECTION AGAINST FAMILIES WITH mtDNA MUTANT MALE PARENTS

Let us assume that there is only random mating, R= 1. In this case, expression (3) becomes

This illustrates that even though there is selection against families with A_{2} male parents, when there is random mating, there is no change in allele frequency because of the maternal inheritance of mtDNA variants.

Next, let us assume that there is R random mating and P positive-assortative mating (R+P= 1). An intuitive way to understand the impact of positive-assortative mating is to assume that P= 1 so that there are only the two mating types A_{1}×A_{1} and A_{2}×A_{2}. As a result, A_{2} females, without any selection against them, are always associated with A_{2} males with selection against them. Therefore, A_{2}×A_{2} matings are selected against, consequently reducing the frequency of mutant A_{2}.

Therefore, with a level R of random mating and P of positive-assortative mating, expression (3) becomes

(4)

Because 1 −s_{m}(Pp_{1}+p_{2}) < 1 −s_{m}p_{2}, , that is, the frequency of the mutant A_{2} decreases. Therefore when P > 0, that is, when there is some positive-assortative mating, the frequency of the mutant A_{2} always decreases.

Now let us examine the effect of positive-assortative mating on the amount of selection against A_{2}. The expected change in allele frequency from expression (4) is

(5a)

Assuming that 1 −s_{m}p_{2}≈ 1, then

(5b)

We can compare this to the reduction in A_{2} to a haploid selection model where 1 and 1 −s are the relative fitness of alleles A_{1} and A_{2}. In this case, the change in allele frequency is

(Hedrick 2011). Solving this expression for s, and assuming that 1 −sp_{2}≈ 1, then the observed selection is

(5c)

In other words,

(5d)

or the product of the amount of selection against families with A_{2} males times the level of positive-assortative mating. [Note that this is identical to the effect found by Wade and Brandvain (2009), fs_{FF}, where they used f to indicate their “inbreeding” level and s_{FF} to indicate selection against families with mutant male parents.] Therefore, when the level of positive-assortative mating is low, then, although there is still selection against the A_{2} mutant, the observed selection against A_{2} may be very small. For example, given that P= 0.01, then s is only 1% of s_{m}.

Using expression (5a), we can determine the expected amount of selection against mutant A_{2}, Figure 1 gives an example when P= 0.1 and s_{m}= 0.1 (solid line). Notice that the amount of selection per generation is small, particularly when p_{2} is low. For example, if p_{2}= 0.1, then Δp_{2} is only −0.00091. From above, this is equivalent to the selection when s= 0.01 for haploid selection.

NEGATIVE-ASSORTATIVE MATING AND SELECTION AGAINST FAMILIES WITH mtDNA MUTANT MALE PARENTS

Let us assume that N is the level of negative-assortative mating and there is R random mating (R+N= 1). An intuitive way to understand the impact of negative-assortative mating is to assume that N= 1 (with polygamy) so that there are only the two mating types A_{1}×A_{2} and A_{2}×A_{1}. As a result, A_{1} females, without any selection against them, are always associated with A_{2} males with selection against them. Therefore, A_{2}×A_{1} matings are selected against, consequently increasing the frequency of maternally inherited mutant A_{2}.

Therefore, with a level R of random mating and N of negative-assortative mating when there is polygamy, expression (3) becomes

(6a)

Because 1 −s_{m}p_{2}R > 1 −s_{m}(p_{2}R+p_{1}N), , and the frequency of the mutant A_{2} increases. Therefore when N > 0, that is, when there is some negative-assortative mating and polygamy, the frequency of the mutant A_{2}, that is detrimental in males, always increases.

The expected change in the frequency of A_{2} is

(6b)

Using the same approach as above, we can show that

(6c)

or that the effect of negative-assortative mating is the product of the amount of selection against families with A_{2} males times the level of negative-assortative mating. Figure 1 gives the expected amount of increase in A_{2} when N= 0.1 and s_{m}= 0.1 (long, broken line). Notice that the amount of increase is exactly the same as the amount of decrease for P= 0.1 and s_{m}= 0.1.

We can use the same approach and show that when there is negative-assortative mating and monogamy [modifying equation (1) to reflect the values in Table 1] that

(7a)

and

(7b)

In this case, when p_{2} is low, then the expected increase in A_{2} frequency is quite low, lower than when there is polygamy, and is lower than the decrease from positive-assortative mating (see Fig. 1, short, broken line).

BOTH POSITIVE- AND NEGATIVE-ASSORTATIVE MATING AND SELECTION AGAINST FAMILIES WITH mtDNA MUTANT MALE PARENTS

One might expect from above that the effects of positive- and negative-assortative (with polygamy) mating would cancel each other out when P=N and the result would be the same as random mating because the effects of the two appear to identical and opposite in sign. To examine this, the change in the frequency of the mutant using expression (3) becomes

(8)

for positive-assortative mating P and negative-assortative mating N (with polygamy).

Obviously, when P=N, the numerator of this expression is 0, there is no expected change in the frequency of A_{2}, and this is indeed equivalent to random mating in its impact. From the numerator of expression (8), when N > P, the frequency of A_{2} is expected to increase and when N < P, the frequency of A_{2} is expected to decrease. The biggest expected change for given values of N and P occurs when s_{m} is the largest and p_{1} and p_{2} are similar in frequency.

Conclusions and Discussion

A model is presented here that explicitly considers the impact of nonrandom mating (positive-assortative and negative-assortative mating) on mtDNA mutants detrimental in males. As found by Wade and Brandvain (2009) and Unckless and Herren (2009), nonrandom mating, given here as positive-assortative mating but called “inbreeding” by them, does allow selection to operate against maternally inherited mtDNA mutants that are detrimental in males and not selected in females. The selective impact reducing the frequency of these mutants is the product of the selection coefficient s_{m} against families with a mutant father and the level of positive-assortative mating P, or s_{m}P, and is equivalent to selection s against a detrimental mutant in haploids.

However, negative-assortative mating (outbreeding) has the opposite impact and results in an increase in the detrimental mutant. This surprising finding results from negative-assortative mating increasing the frequency of matings between A_{1} males and A_{2} females and then these A_{2} females subsequently contribute mutant mtDNA to their progeny because it was not detrimental in these families that had an A_{1} father. The increase when there is negative-assortative mating and polygamy is identical to the decrease when there is positive-assortative mating. When there is negative-assortative mating and monogamy, the expected increase in mutants at low frequencies is substantially lower than with polygamy.

As an extension, if there was paternal mtDNA transmission, as in some animals and plants, or a high rate of paternal leakage, then negative-assortative mating would select for detrimental female-specific mtDNA variants. Or in general, if a detrimental uniparentally inherited variant can “avoid itself” through negative-assortative mating, then it will be favored by selection.

Engelstädter and Charlat (2006) examined a cytoplasmic genetic model with deleterious effects in males and “outbreeding” in the form of sib-mating avoidance. They showed theoretically that in this case such “spiteful” cytoplasmic elements can reduce the number of offspring produced by males and are selected for. The model of negative-assortative mating provided here generalizes their findings and shows the potential impact of outbreeding on increasing detrimental mtDNA variants.

The treatment here makes a number of assumptions. For example, selection is assumed to act on male reproduction or male fertility. If selection is assumed to influence male viability, the impact is different and can result in a stable polymorphism as discussed by Unckless and Herren (2009) (for related models, see Kidwell et al. 1973; Strobeck 1979). In addition, it is assumed here that there is no association between a particular mtDNA type and sex, assortative mating is based on mtDNA types, and that generations are nonoverlapping.

Although nonrandom mating in the form of positive-assortative mating may result in a decrease in the frequency of mtDNA mutants detrimental in males, negative-assortative mating can result in an increase in the frequency of mtDNA mutants and counter this effect. To predict the overall impact of nonrandom mating on such mutants, the level of the different types of nonrandom mating needs to be determined.

Gemmell and Allendorf (2001) and Gemmell et al. (2004) in their discussion of population viability suggested that detrimental male-specific mtDNA variants that increase in frequency because of maternal inheritance may increase the probability of population extinction depending upon the mating system of the species. In addition, an increase in the variance of male contributions may also reduce the male effective population size, thereby potentially reducing nuclear genetic variation. Positive-assortative mating should reduce the frequency of these variants and reduce the impact on extinction and male effective population size. On the other hand, negative-assortative mating may increase the frequency of these variants, thereby increasing the probability of extinction and reducing the male effective population size. In addition, endangered species management should be careful to not introduce individuals from another population that has detrimental male-specific mtDNA variants (Smith et al. 2010).

Associate Editor: C. A. Buerkle

ACKNOWLEDGMENTS

This research was partially supported by the Ullman Professorship. I appreciate the comments of F. Allendorf, Y. Brandvain, S. Frank, J. Herren, R. Unckless, and M. Wade on the topic or an earlier version of the manuscript.