How species evolve depends on the communities in which they are embedded. The idea that species exhibit traits shaped by collections of other species that co-occur with them received growing attention from the late 1970s through to the present. D. S. Wilson (1976) stated that ‘… every effect of a species on a community will loop back to influence the species itself, either positively or negatively.’ The term ‘diffuse coevolution’ was introduced by Janzen (1980) in his note ‘When is it coevolution?’ to describe the idea that selection on traits often reflects the actions of many community members, as opposed to pairwise interactions between species. Fox (1981) suggested that plants might exhibit generalized adaptations to cope with a suite of attacking herbivores, rather than having traits that were result of a one-on-one coevolutionary arms race. These ideas were further clarified by Gould (1988) by focusing on a variety of ecological and genetic mechanisms that might lead to diffuse coevolution in response to selection from multiple species. Manipulative experiments assessing the effects of multiple herbivores on each other's patterns of host plant use (Faeth, 1986) and on their effects singly vs in combination on plant growth, reproduction, and survival (Strauss, 1991) suggested the possibility that multiple species interact with one another to create selective regimes that may not be well understood by knowing selection resulting from one-on-one, pairwise interactions. In one of the first papers taking a quantitative evolutionary approach, Simms (1990) considered analytical methods to measure the indirect selection imposed by multiple herbivores on plant resistance to a focal herbivore. Subsequently, in a pair of seminal papers, Hougen-Eitzman & Rausher (1994) and Iwao & Rausher (1997) developed carefully delineated criteria to identify when selection is diffuse rather than pairwise.
Diffuse coevolution, diffuse selection, and diffuse evolution
In spite of the theoretical and conceptual attention paid to diffuse coevolution, to date, few studies have demonstrated reciprocal changes in species in response to selection caused by communities of differing composition. Exceptions to this statement lie in the character displacement literature, in which researchers have shown that species exhibit reciprocal shifts in trait values in response to the presence or absence of other species (Schluter et al., 1985; Benkman, 2003; Knouft, 2003). The community context of diffuse coevolution is probably being most closely addressed in a set of studies by Benkman and colleagues on crossbills, red squirrels, insects, and lodgepole pines (Benkman, 1996, 1999; Benkman et al., 2001, 2003). Pine cone morphology changes in response to directional selection from all cone predators – moths, red squirrels, and crossbills, and each herbivore in isolation results in a different pattern of selection on cone morphology (pair-wise selection). Divergent selection on cone morphology between areas with different cone predators is congruent with the geographic patterns of cone variation. In areas where squirrels are absent, there are reciprocal evolutionary responses in the crossbill population in response to cone traits (cone morphology is demonstrated as an agent of selection on bill shape). In areas with both squirrel and crossbill cone predators, cone traits reflect primarily the selective effects of squirrel predation; bill morphology of crossbills that co-occur with squirrels differs from that of birds in areas lacking squirrel competitors (Benkman et al., 2001). Benkman and colleagues are now adding in the complicating factors of how traits change with additional seed predators like cone-feeding moths, which select for yet a different type of cone morphology (Benkman, 2003), and how these cone changes may affect the evolution of bill structure in the crossbills. This set of studies incorporates shifts in community composition, community-dependent selection, and evolutionary responses through trait shifts in both the plant and animal interactors.
By contrast to the Benkman studies, the bulk of the literature concerned with effects of communities focuses on diffuse evolution (as opposed to coevolution). These studies examine how a focal (usually consumed) species is affected by altered community composition (usually of consumer species) [e.g. (Galen & Cuba, 2001; Relyea, 2002a,b; Stinchcombe & Rausher, 2002; Ghalambor et al., 2003)]. For example, tadpole tail shape and color, as well as foraging behavior, correlate with the species composition and abundance of amphibian vs insect predators (Relyea, 2002a,b). Tadpole morphology and behavior also change in response to competitive environment (Relyea, 2002b). Moreover, predation and competition have interactive effects, such that the magnitude of tadpole responses to predators depends on the competitive environment, and vice versa (e.g. responses in tail shape to predators are larger under low competition than under highly competitive regimes (Relyea, 2004)). Many of these shifts in morphological and behavioral traits are a result of phenotypic plasticity (which presumably itself has a genetic basis); in addition, some predator avoidance tail traits in tadpoles have a nonplastic genetic basis as well, and patterns of tail trait variation are consistent with local adaptation by tadpoles to predator community composition (Van Buskirk & McCollum, 1999; Relyea, 2002b).
Another area of study in which there is substantially similar work, and that will be the subject of this review, focuses on the response of plants to multiple consumers/interactors. To date, all of these studies have examined the response of the consumed plant species, and not reciprocal evolutionary change in the consumers. As such, the remainder of the examples discussed will reflect diffuse selection and the response to diffuse selection (diffuse evolution), but do not address diffuse coevolution, which, as we interpret that term, requires reciprocal evolutionary change by multiple species. To our knowledge, there are no documented cases of changes in selection on plant consumers caused by interactions with other species, other than the studies of Benkman and colleagues described above. Selection pressures on, for instance, mandible size or detoxification ability of a later-feeding herbivore might differ depending on the community composition of the herbivore species feeding before it on an inducible, shared host plant. Similarly, we are unaware of studies showing how the evolution of one pollinator species is affected by the presence or absence of other species. This area is in need of more experimental attention, as it is likely that community composition shapes traits in consumer, as well as consumed, species (Inouye & Stinchcombe, 2001).
Evaluating the criteria for diffuse selection and diffuse evolution
Here, we examine diffuse selection and diffuse evolution in plant–animal systems and clarify theoretical and empirical approaches to their study. As mentioned before, two papers outline the original criteria delineating when evolution is expected to reflect pairwise interactions between species (Hougen-Eitzman & Rausher, 1994; Iwao & Rausher, 1997); violations of these criteria mean that evolution is diffuse. It should be noted that these authors were particularly concerned with herbivory, and the special case in which each herbivore is exerting selection on a different plant trait (i.e. plant resistance to that herbivore). These criteria are quoted below:
- 1The susceptibilities (resistances) to different herbivores are genetically uncorrelated (Paraphrased for broader communities: Traits important to interactions with multiple species are not genetically correlated with one another.)
- 2The presence/absence of one herbivore does not affect the amount of damage caused by other herbivores (Paraphrased to be more general: The presence or absence of one community member does not cause a plastic change within a generation in a trait involved in interactions with other community members).
- 3The impact of one herbivore on plant fitness does not depend on the presence/absence of other herbivores. (Paraphrased to be more general: The impact of an interactor on a focal species’ fitness does not depend on the presence/absence of other species.)
Here, we argue that the way in which Criteria 2 and 3 were originally worded has led to a misunderstanding of the ideas in some of these papers. In particular, because the wording of these criteria does not obviously include selection (but see Stinchcombe & Rausher, 2002) – that is, the relationship between relative fitness and specific phenotypic traits – it has reinforced the misconception that measuring responses to community composition in terms of interaction strength or mean fitness is sufficient to document diffuse selection or diffuse evolution (Strauss, 1991; Karban & Strauss, 1993; Wise & Sacchi, 1996; Parmesan, 2000; Herrera et al., 2002; Hufbauer & Root, 2002; Adler, 2003; Anderson & Paige, 2003). In this discussion, we show why simply examining nonadditive effects of multiple species on fitness are not sufficient to demonstrate diffuse selection. In addition, we rephrase and broaden the criteria above such that satisfying these criteria would constitute evidence of diffuse selection or evolution.
There are a number of studies that have addressed the evidence for each of these criteria as originally stated and they have been reviewed more extensively elsewhere (Rausher, 1996; Strauss & Irwin, 2004). Our main point, and one that has been made before (Juenger & Bergelson, 1998; Inouye & Stinchcombe, 2001; Stinchcombe & Rausher, 2002), but bears repeating and further clarification, is that studies that take Criteria 2 and 3 as they were originally worded do not provide information about selection or evolution in a community context. It should be noted, however, that the mathematical descriptions of the criteria and the methods outlined in Iwao & Rausher (1997) can and do tell us about selection.
Criterion 2 describes the dependency of interactions on the presence or absence of other species in the community. Undoubtedly, such dependencies are ubiquitous. For example, the induced defense literature is replete with examples in which attack by one herbivore alters the use of plants by subsequently feeding herbivores (reviewed in Karban & Baldwin, 1997). Also, the presence of cross-talk between induction of pathogen resistance and subsequent effects on herbivore resistance, and vice versa, represents a growing body of literature showing the importance of community context in determining the strength of interactions (Felton & Korth, 2000; Heil & Bostock, 2002; Thaler et al., 2002). Strauss & Irwin (2004) recently reviewed the effects that mutualists have on each other via use of a shared plant resource. For example, scent-marking of flowers by pollinators reduces or attracts visitation to flowers by subsequently foraging pollinators (Guirfa, 1993; Goulson et al., 2000). Similarly, depletion of nectar rewards or seeds by one species can reduce the use of plants by other pollinators or seed dispersers. The interdependency of interactions is suggestive of the kinds of selective effects that may be present in different community contexts, but without taking accompanying measurements of how species interactions affect selection on specific traits in the focal species, one cannot gain an understanding of how altered interactions translate into altered selective regimes. Because selection is defined as the relationship between fitness and a phenotypic trait (Conner & Hartl, 2004), both traits and fitness need to be measured.
The next criterion (3) states that the impact of one interactor on plant fitness is not dependent on the presence or absence of other species. This criterion is the one that appears to be the most commonly misunderstood. Hougen-Eitzman & Rausher (1994) measured plant traits (resistances to different herbivores) and plant fitness; however, their analyses only addressed the nonindependence of damage and the nonadditive effects of herbivores on plant fitness. They did not estimate selection in their different community treatments. By contrast, Iwao & Rausher (1997) address the same criteria within the framework of selection gradients and differentials that relate relative fitness to trait values. However, the verbal description of the criteria remained unchanged and thus has resulted in the misguided notion that documenting the nonadditive effects of multiple species on mean fitness of a focal species is sufficient to show diffuse selection. Nonadditive effects of multiple interactors on fitness will not lead to diffuse selection unless the relationship between relative fitness and at least one trait is altered along with the change in fitness.
|Box 1 Definitions (many modified from Conner & Hartl, 2004)|
|• Relative fitness: absolute fitness divided by the population mean fitness.|
|• Selection: a relationship between relative fitness and a phenotypic trait within a generation.|
|• Linear selection gradient: a measure of the strength of directional selection, estimated as the slope of the linear regression of relative fitness on a trait.|
|• Nonlinear (quadratic) selection gradient: a measure of the degree of curvature in the relationship between relative fitness and a trait.|
|• Response to selection: a change in allele frequency or mean phenotype from one generation to the next caused by selection.|
|• Diffuse evolution: when selection or the response to selection imposed by one species on another is dependent on the presence or absence of other species in the community (paraphrased from Gould, 1988). (Diffuse coevolution, by extension, is when selection imposed reciprocally by one species on another is dependent on the presence or absence of other species.)|
|• G-matrix: a square matrix with additive genetic variances for the traits on the diagonal and additive genetic covariances on the off-diagonal (synonym: additive genetic variance-covariance matrix). Heritabilities and genetic correlations are standardized genetic variances and covariances respectively.|
Most traits are continuously distributed, so the techniques of quantitative genetics are appropriate for measuring selection on traits imposed by community members. To review, adaptive evolution (and coevolution) are most commonly studied using a pair of closely related equations (Lande, 1979; Lande & Arnold, 1983; Arnold & Wade, 1984 a,b ):
The first equation is the univariate case and the second is the analogous equation for the multivariate case, in which more than one trait is considered simultaneously; the latter is more useful because individual traits do not evolve in isolation. In both equations, the response to selection (R or ) is the change in the mean value of a trait across one generation. This is short-term phenotypic evolution, which is the product of genetic variance (h2 or G) and the strength of selection (S or β). The most important difference between these equations is that G also contains the genetic covariances among traits. Therefore evolution is diffuse if changes with the addition of a second interacting species, and this can occur either through changes in selection (β) or the genetic variance/covariance matrix (G).
Note that changes in population mean fitness can occur as a result of interactions among species without changing selection on a trait (the slope of the line; Fig. 1b). For example, individual or family mean fitness may be reduced by the presence of an additional interactor (Species A + B in Fig. 1a) but, if fitness is reduced the same amount for each individual or family, this ecological interaction will not change the slope of the relationship between relative fitness and phenotype, and thus will not alter selection (Fig. 1b). The slope of the line relating the relative fitness of individuals to a trait is not necessarily affected by community composition, even though the effects of Species A and B together on population mean fitness of the focal species are nonadditive.
Changes in fitness may also not result in diffuse selection if community members select on different traits. For example, if one herbivore selects for increased latex production, and another herbivore selects for increased trichome number, the presence of both herbivores at once will not alter the direct selection imposed by either herbivore alone (nor the response to selection if the traits are genetically uncorrelated). Thus, fitness may be decreased when both herbivores are present, but the evolution of each trait in response to each herbivore individually will not be altered because the herbivores are selecting on two different traits. Therefore, there is no reason to expect that fitness effects of multiple interactors necessarily result in diffuse selection. In summary, changes in fitness of a focal species in response to shifting community composition provide no information about selection or about whether evolution is diffuse or not. This point is illustrated with empirical examples from the literature below.
A few studies have addressed whether selection is diffuse and have incorporated measurements of selection on traits, knowledge of the relative fitnesses of families (a substitute for genotypes) or individuals, and mean fitness. Below, we review three of these studies and their results in light of effects of community composition on mean fitness and on selection. All of these studies use fully crossed factorial designs in which treatments are the presence or absence of one herbivore in the presence or absence of another herbivore. Selection is measured in each treatment by regressing relative fitness on the trait of interest.
Juenger & Bergelson (1998) examined interactions between three different herbivores – caterpillars, seed flies, and clipping (deer herbivory simulation) – in terms of their effects on both plant fitness and selection on phenology. For seed production, they found a significant two–way interaction between the effects of caterpillars and flies, but no three–way interaction including clipping. In other words, only caterpillars and flies interacted to create nonadditive effects on seed production (fitness). However, when these authors compared directional selection on a trait, flowering phenology, by flies and caterpillars individually and together, there was no significant fly × caterpillar × phenology interaction; that is, the slope of the relationship between relative fitness and flowering phenology did not differ among the fly and caterpillar treatments. Thus, flies and caterpillars do not exert diffuse directional selection on phenology even though they have nonadditive effects on plant fitness. By contrast, analysis of quadratic selection gradients showed that the caterpillar × fly × clip × phenology2 interaction was significant, indicating diffuse nonlinear selection on phenology when all three herbivores use plants. Recall that there was no significant three–way interaction in the effects of the three herbivores on seed production (fitness). Thus, in both cases, interactions for mean fitness did not correspond to differences in selection. Therefore, unless we know the relationship between a trait and relative fitness in response to experimental treatments (as opposed to simply changes in mean fitness in the treatments), we can say nothing about the nature of selection or diffuse evolution in these treatments.
Stinchcombe & Rausher (2002) measured how selection on tolerance and resistance to deer herbivory changed in the presence and absence of other herbivores. They found that adding insect herbivores did not alter the fitness impacts of deer herbivory, but they did find significantly different strengths of selection on the traits tolerance and resistance to deer herbivory in the presence or absence of insect herbivores. Selection to decrease tolerance to deer herbivory and to increase resistance to deer was stronger when insect herbivores were present than when insects were excluded. Thus, it is possible to find no interaction between different species of herbivore in mean fitness, and yet find significant differences in selective regimes in the presence and absence of that herbivore.
In a third example, Pilson (1996) examined how the presence or absence of diamondback moths and flea beetles altered selection on flea beetle damage. She found a significant interaction between the effects of flea beetles and diamondbacks on plant fitness. She also found that selection on flea beetle damage at high and low moth densities was in different directions, so that at low moth densities, plants with low flea beetle damage are more fit, and at high moth densities, plants with high levels of flea beetle damage are more fit. Therefore, in this study a significant interaction between the fitness effects of flea beetle and moth herbivory was coupled with diffuse selection on flea beetle resistance, but no reason was found to expect a causal relationship.
These studies are to be commended because they incorporate all the needed elements to assess diffuse selection (Iwao & Rausher, 1997; Tiffin, 2002). Many studies examining multiple interactors can only provide an ecological understanding of the effects of complex interactions. For example, several studies consider the nonadditive effects of multiple interactors on fitness and results are subsequently placed in the context of selection on plant traits; some of these papers directly cite the criteria outlined in Hougen-Eitzman & Rausher, 1994 and Iwao & Rausher, 1997 (Strauss, 1991; Karban & Strauss, 1993; Wise & Sacchi, 1996; Parmesan, 2000; Herrera et al., 2002; Hufbauer & Root, 2002; Adler, 2003; Anderson & Paige, 2003). Other studies have documented both changes in traits and changes in fitness when multiple interactors are involved, but have not focused on how the relationship between a trait and relative fitness (i.e. selection) is altered with the presence or absence of different community members (Hougen-Eitzman & Rausher, 1994; Agrawal & Van Zandt, 2003).
These studies may provide insights into the importance of community composition on plant population dynamics, but tell us little about selection or evolution, for the reasons described above. For example, Herrera et al. (2002) found nonadditive effects of browsing ungulate herbivores and pollinators on fitness of the perennial plant, Helleborus foetidus; pollinators had positive effects on seedling recruitment (fitness), but only in the absence of vertebrate herbivory. These authors show dramatic differences in plant population size among treatments in which the community composition was manipulated factorially; these strong ecological effects attest to the importance of nonadditive effects of species composition (i.e. communities) on plant population dynamics. Effects of community composition on population size may have evolutionary implications, since inbreeding depression, drift and other important evolutionary processes can be driven by population size. However, effects on mean plant fitness and population size and cannot tell us about selection on traits involved in herbivore defense or pollinator attraction in these treatments.
Modified criteria for diffuse selection and evolution
Starting from the baseline of the earlier papers, we rephrase and expand criteria 2 and 3 for documenting diffuse selection and evolution to reflect more explicitly the importance of selection on traits. We then describe some experimental designs that can test for diffuse selection and diffuse evolution. In order to think about diffuse evolution it is best to return to the definition of pairwise evolution:
Pairwise coevolution occurs when selection imposed reciprocally by the plant and each herbivore and the response to that selection are not influenced by the presence or absence of the other herbivore species (Gould, 1988). To generalize this definition to other kinds of interactions, the basic criterion for pairwise evolution is that the evolutionary rate and trajectory of a specific trait that is evolving in response to selection by an interacting species is not affected by the presence or absence of other interacting species. This criterion can be violated if:
- 1The second species causes correlated responses to selection in the focal trait caused by genetic covariances between traits. So, one way for evolution to be diffuse is if traits under selection by different species are genetically correlated with one another.
- 2The total strength or direction of selection on the trait is altered by the second interacting species, either in an additive or nonadditive manner. So, another mode of diffuse evolution is when the presence or absence of other species causes a change in the strength or direction of selection on a trait relative to the selection exerted by the first species. This criterion includes multiple interactors selecting in opposing directions on the same trait. Note that a change in selection will lead to a change in response to selection if there is additive genetic variance for the trait (Eqn. 1).
- 3The presence of the second species alters the G-matrix; that is, the expression of genetic variance for the focal trait or genetic covariances between the focal trait and other traits under selection change in the presence of a second species. This describes a genotype by environment interaction, where the environments are the presence and absence of the second interacting species.
In other words, evolution is diffuse if the response to selection by one interacting species alone is altered by the presence of a second interacting species. The first criterion is an extension of Iwao and Rausher's criterion one, the second is a combination of their criteria two and three and includes the special case in which the same trait is affected by multiple community members, and the third is an extension of Stinchcombe & Rausher's (2002) third criterion. For the criteria we have substantially altered (the second and third), we add discussion below.
Criterion 2: We clarify that the effects of interactors can be additive or nonadditive and still satisfy the definition of diffuse evolution (the evolutionary trajectory of a trait will depend on the presence or absence of other species in the community). The most obvious example occurs when multiple interactions are affected by the same trait (called ‘ecological pleiotropy’ in Strauss & Irwin, 2004). In this case, the presence or absence of species with conflicting effects on trait evolution will determine how a trait in the focal species evolves.
There are several examples of opposing selection on a single trait in the plant–animal literature: increased corolla flare increases the use of Polemonium viscosum flowers by mutualist pollinators, but also by antagonistic, nectar-thieving ants (Galen, 1999). Large floral displays attract pollinators, but also seed predators that use flower buds for oviposition (Brody, 1992; Herrera, 2000). Secondary compounds confer resistance to generalist herbivores, but increase susceptibility to specialists (Van Dam et al., 1995; Agrawal & Sherriffs, 2001; Kliebenstein et al., 2002). In each of these cases, traits will be under fluctuating selection determined by the presence and abundance of different community members, and thus evolution is diffuse.
The distinction between additive and nonadditive selective effects is worthy of consideration and investigation, however, because nonadditive effects represent emergent properties of communities (Inouye & Stinchcombe, 2001). If the selective effects of multiple species on a trait are additive, then one could predict how a trait might evolve in any community assemblage simply by adding together the effects of each species on the trait when that species occurs alone with the focal species. For example, with the lodgepole pine cones, if effects of predators were additive, one could predict cone shape in communities where both squirrels and crossbills were present by adding together the selection gradients on cone characters imposed by each predator in the habitat where they occur alone. Such a situation could be considered ‘multiple, pairwise’ evolution. By contrast, nonadditive selective effects of squirrels and birds on cone characters would mean that our predictions of the effects of both species together based on pairwise selection gradients would be incorrect, and that evolutionary trajectories of cone traits could only be determined by measuring the relationship between traits and fitness in the presence of all community members. Nonadditive effects may reflect a range of ecological interactions including trait-mediated or density-mediated indirect effects, as well as direct interactions like interference competition (Peckarsky & McIntosh, 1998). Since the evolutionary trajectories of traits are modified by the presence or absence of other species, regardless of whether the effects of multiple species are additive or not, both modes of selection will result in diffuse evolution as it is defined by Gould (1988).
Criterion 3. Stinchcombe & Rausher (2002) added the following criterion for pairwise coevolution:
‘The genes contributing to variation in resistance (or tolerance) to each enemy must be the same in the presence and absence of other enemies.’ We extend this in our third criterion by stating that evolution is diffuse if changes in community composition alter genetic variances or covariances (the G-matrix) for a trait under selection by one interactor. Within one population and generation, there are no changes in allele frequencies, so any observed genetic change is due to differences in gene expression caused by the change in the environment. It is important to remember that genetic variances and covariances are for phenotypic traits, and therefore these variances and covariances can be altered by changes in allele frequencies (evolution) or by changes in gene expression influenced by the environment. For example, Conner et al. (2003) reported large changes in the expression of additive variance and covariance for six floral traits in the same population of wild radish when plants were grown in the field vs the glasshouse. Stinchcombe & Rausher (2002) found that tolerance to deer herbivory in the presence of insect herbivores and pathogens was genetically uncorrelated with tolerance in the absence of other enemies, which is genotype by environment interaction for tolerance due to differences in the expression of tolerance between the environments with and without other enemies. Changes in the G-matrix will alter the rate and possibly the trajectory of adaptive evolution in response to selection.
Experimental designs that test the importance of community composition to selection on traits, and/or evolution in, a focal species
Several experimental designs can provide evidence for diffuse selection or diffuse evolution (Iwao & Rausher, 1997; Inouye & Stinchcombe, 2001). It is important to distinguish between the within-generation effects of selection and the between-generation response that is phenotypic evolution. Selection estimates give us an idea of the relationship between relative fitness and trait variation within a generation, while between-generation evolution tells us how a population responds to such selection based on the G-matrix.
The simplest community is a three-species community, although experimenters also often manipulate whole guilds (like insect herbivores or pathogens) when they use broad-acting treatments like insecticides or fungicides to remove the effects of a suite of community members. For the purposes of these examples, we will take the simplest approach to assess how the presence/absence of two different species affects selection on a trait of a third species.
A basic design that would indicate whether diffuse selection is occurring does not necessarily require elaborate methods. Using a Lande & Arnold (1983) approach, one could randomly assign members of a population to three different community treatments: Species 1 only, Species 2 only, and Species 1 and 2. For each individual, one would measure a trait or traits suspected to be important in the interactions between these species, as well as the fitness of each individual (e.g. lifetime offspring production or a component of this such as lifespan or fecundity). To analyze an experiment with this design, the following analysis of covariance (ancova) model is used:
In this model, trait value is a continuous variable, and the resulting slope measures selection averaged over all community types. Community is a categorical treatment with three levels – Species 1 only, Species 2 only, and Species 1 and 2 together; this tests for differences in mean fitness among the three treatment groups. The interaction term tests for differences in slopes among treatment groups, testing whether the strength of selection on the trait is significantly affected by community composition. Diffuse nonlinear selection can be studied by adding squared trait values and the interaction between these and the community. This approach was used in the study by Juenger & Bergelson (1998).
If the interaction term were significant, then one could perform a series of a priori contrasts within the interaction term comparing the slopes (selection gradients between the single species and two-species treatments). The two contrasts of each single species treatment with the two-species treatment tests whether adding the second species alters the selection exerted by each species alone. The contrast of the two single species treatments together with the two-species treatment tests for nonadditive effects of species composition on selection on the measured trait, because a significant result indicates that the two-species selection gradient is not just the average of the one-specie gradients (Fig. 1c). This purely phenotypic design would not indicate whether a response to selection would occur, since it provides no information about the heritability of the trait.
Another design can include family structure, and provides information about the genetic variances and covariances among traits (Criteria 1 and 3) as well as the importance of community composition to the evolution of component species. In this case, a standard quantitative genetic nested half-sibling design should be implemented, with individuals from each full-sibling family split at random among community treatments (three levels as above: Species 1, Species 2 and Species 1 and 2 together). This is the standard design for testing genotype by environment interaction (G × E) in sexual species (Via, 1994). Ideally a number of traits would be measured, with traits chosen based on knowledge of the biology of the interaction. Several different analyses could then be undertaken, including the partitioning of variation in selection gradients into pairwise and interactive components as outlined in Iwao & Rausher (1997). In addition, an analysis similar to (1) described above could also be undertaken, except using breeding values (or family means) for traits and fitness, instead of individual values (Rausher, 1992). Finally, the entire G-matrix for traits under selection by the interacting species can be compared among treatment groups (examples of statistical techniques in Conner, 2002); differences are evidence for diffuse evolution under criterion 3.
If a study such as those outlined above found no evidence of selection on a trait, or no evidence of differences in selection, it would be wrong to conclude that diffuse selection is not occurring in the system. The only valid conclusion we can draw from such a result is that diffuse selection is not acting on the measured traits. The ‘wrong’ traits may have been selected for study, or diffuse selection may really not be occurring. One other advantage of incorporating family structure into an experimental design is the ability to test for a family * treatment interaction on the fitness measure. A significant interaction would mean that the relative fitness of families changes across treatments (i.e. with shifting community composition), and would also be evidence for diffuse evolution, since the relative fitness of genotypes depends on the presence or absence of other species. However, this result does not shed any light on which traits/interactions are important in this shift in relative fitness of genotypes; it would suggest, however, that it would be worth considering different traits in future experiments.