#### POSITIVE-ASSORTATIVE MATING AND SELECTION AGAINST FAMILIES WITH mtDNA MUTANT MALE PARENTS

Let us assume that there is only random mating, *R*= 1. In this case, expression (3) becomes

This illustrates that even though there is selection against families with *A*_{2} male parents, when there is random mating, there is no change in allele frequency because of the maternal inheritance of mtDNA variants.

Next, let us assume that there is *R* random mating and *P* positive-assortative mating (*R*+*P*= 1). An intuitive way to understand the impact of positive-assortative mating is to assume that *P*= 1 so that there are only the two mating types *A*_{1}×*A*_{1} and *A*_{2}×*A*_{2}. As a result, *A*_{2} females, without any selection against them, are always associated with *A*_{2} males with selection against them. Therefore, *A*_{2}×*A*_{2} matings are selected against, consequently reducing the frequency of mutant *A*_{2}.

Therefore, with a level *R* of random mating and *P* of positive-assortative mating, expression (3) becomes

- (4)

Because 1 −*s*_{m}(*Pp*_{1}+*p*_{2}) < 1 −*s*_{m}p_{2}, , that is, the frequency of the mutant *A*_{2} decreases. Therefore when *P* > 0, that is, when there is some positive-assortative mating, the frequency of the mutant *A*_{2} always decreases.

Now let us examine the effect of positive-assortative mating on the amount of selection against *A*_{2}. The expected change in allele frequency from expression (4) is

- (5a)

Assuming that 1 −*s*_{m}p_{2}≈ 1, then

- (5b)

We can compare this to the reduction in *A*_{2} to a haploid selection model where 1 and 1 −*s* are the relative fitness of alleles *A*_{1} and *A*_{2}. In this case, the change in allele frequency is

(Hedrick 2011). Solving this expression for *s*, and assuming that 1 −*sp*_{2}≈ 1, then the observed selection is

- (5c)

In other words,

- (5d)

or the product of the amount of selection against families with *A*_{2} males times the level of positive-assortative mating. [Note that this is identical to the effect found by Wade and Brandvain (2009), *fs*_{FF}, where they used *f* to indicate their “inbreeding” level and *s*_{FF} to indicate selection against families with mutant male parents.] Therefore, when the level of positive-assortative mating is low, then, although there is still selection against the *A*_{2} mutant, the observed selection against *A*_{2} may be very small. For example, given that *P*= 0.01, then *s* is only 1% of *s*_{m}.

Using expression (5a), we can determine the expected amount of selection against mutant *A*_{2}, Figure 1 gives an example when *P*= 0.1 and *s*_{m}= 0.1 (solid line). Notice that the amount of selection per generation is small, particularly when *p*_{2} is low. For example, if *p*_{2}= 0.1, then Δ*p*_{2} is only −0.00091. From above, this is equivalent to the selection when *s*= 0.01 for haploid selection.

#### NEGATIVE-ASSORTATIVE MATING AND SELECTION AGAINST FAMILIES WITH mtDNA MUTANT MALE PARENTS

Let us assume that *N* is the level of negative-assortative mating and there is *R* random mating (*R*+*N*= 1). An intuitive way to understand the impact of negative-assortative mating is to assume that *N*= 1 (with polygamy) so that there are only the two mating types *A*_{1}×*A*_{2} and *A*_{2}×*A*_{1}. As a result, *A*_{1} females, without any selection against them, are always associated with *A*_{2} males with selection against them. Therefore, *A*_{2}×*A*_{1} matings are selected against, consequently increasing the frequency of maternally inherited mutant *A*_{2}.

Therefore, with a level *R* of random mating and *N* of negative-assortative mating when there is polygamy, expression (3) becomes

- (6a)

Because 1 −*s*_{m}p_{2}*R* > 1 −*s*_{m}(*p*_{2}*R*+*p*_{1}*N*), , and the frequency of the mutant *A*_{2} increases. Therefore when *N* > 0, that is, when there is some negative-assortative mating and polygamy, the frequency of the mutant *A*_{2}, that is detrimental in males, always increases.

The expected change in the frequency of *A*_{2} is

- (6b)

Using the same approach as above, we can show that

- (6c)

or that the effect of negative-assortative mating is the product of the amount of selection against families with *A*_{2} males times the level of negative-assortative mating. Figure 1 gives the expected amount of increase in *A*_{2} when *N*= 0.1 and *s*_{m}= 0.1 (long, broken line). Notice that the amount of increase is exactly the same as the amount of decrease for *P*= 0.1 and *s*_{m}= 0.1.

We can use the same approach and show that when there is negative-assortative mating and monogamy [modifying equation (1) to reflect the values in Table 1] that

- (7a)

and

- (7b)

In this case, when *p*_{2} is low, then the expected increase in *A*_{2} frequency is quite low, lower than when there is polygamy, and is lower than the decrease from positive-assortative mating (see Fig. 1, short, broken line).

#### BOTH POSITIVE- AND NEGATIVE-ASSORTATIVE MATING AND SELECTION AGAINST FAMILIES WITH mtDNA MUTANT MALE PARENTS

One might expect from above that the effects of positive- and negative-assortative (with polygamy) mating would cancel each other out when *P*=*N* and the result would be the same as random mating because the effects of the two appear to identical and opposite in sign. To examine this, the change in the frequency of the mutant using expression (3) becomes

- (8)

for positive-assortative mating *P* and negative-assortative mating *N* (with polygamy).

Obviously, when *P*=*N*, the numerator of this expression is 0, there is no expected change in the frequency of *A*_{2}, and this is indeed equivalent to random mating in its impact. From the numerator of expression (8), when *N* > *P*, the frequency of *A*_{2} is expected to increase and when *N* < *P*, the frequency of *A*_{2} is expected to decrease. The biggest expected change for given values of *N* and *P* occurs when *s*_{m} is the largest and *p*_{1} and *p*_{2} are similar in frequency.