Habitat specialization plays an important role in the creation and loss of biodiversity over ecological and evolutionary time scales. In California, serpentine soils have a distinctive flora, with 246 serpentine habitat specialists (i.e., endemics). Using molecular phylogenies for 23 genera containing 784 taxa and 51 endemics, we infer few transitions out of the endemic state, which is shown by an analysis of transition rates to simply reflect the low frequency of endemics (i.e., reversal rates were high). The finding of high reversal rates, but a low number of reversals, is consistent with the widely hypothesized trade-off between serpentine tolerance and competitive ability, under which serpentine endemics are physiologically capable of growing in less-stressful habitats but competitors lead to their extirpation. Endemism is also characterized by a decrease in speciation and extinction rates and a decrease in the overall diversification rate. We also find that tolerators (species with nonserpentine and serpentine populations) undergo speciation in serpentine habitats to give rise to new serpentine endemics but are several times more likely to lose serpentine populations to produce serpentine-intolerant taxa. Finally, endemics were younger on average than nonendemics, but this alone does not explain their low diversification.

Ecological specialization in habitat use is remarkably common in nature (Futuyma and Moreno 1988; Stevens 1989; Brown 1995; Gaston and Blackburn 2000). In areas of high environmental heterogeneity, habitat specialists, defined as taxa that use a subset of available habitats, make major contributions to species diversity by promoting turnover in species composition (Fine et al. 2005; Kraft et al. 2008). In addition to shaping ecological patterns, habitat specialization likely plays an important role in the gain and loss of biodiversity on evolutionary time scales. However, basic questions about the evolutionary origins and consequences of habitat specialization remain unanswered. The recent accumulation of DNA sequence data makes it possible to test explicitly for directional evolutionary pathways and differential diversification associated with habitat specialists using phylogenetic methods.

Habitat specialization could have contrasting evolutionary consequences (Berenbaum 1996). When habitat specialization is associated with the exploitation of formerly empty niches, speciation and even adaptive radiations may result (Losos et al. 1998; Schluter 2002; Grant and Grant 2007). In addition, environmental heterogeneity coupled with patchily distributed habitats can limit gene flow, promoting local adaptation and subsequent diversification under certain conditions (Ackerly 2003). However, the restricted habitat availability associated with specialists compounded with habitat insularity can also lead to an “evolutionary dead-end” (Cope 1896; Takebayashi and Morrell 2001). If habitat patches are small, isolated, rare, or of poor quality, then small populations with low genetic variation and restricted geographic ranges could face increased extinction risk (Berenbaum 1996; Losos et al. 1998; Schluter 2002; Ackerly 2003; Grant and Grant 2007).

Here, we examine whether directional evolutionary pathways and predictable patterns of lineage diversification underlie habitat specialization in plants on serpentine soils in California. Serpentine soils are among the most striking examples of how steep ecological gradients can promote habitat specialization. Plants that grow on serpentine must adapt to low levels of essential macronutrients, elevated levels of heavy metals and magnesium, and low water-holding capacity, among other challenges (Brady et al. 2005; O’Dell and Claassen 2006a,b; O’Dell et al. 2006). A significant amount of plant diversity is associated with these extremely stressful conditions. In the California flora alone, serpentine habitat specialists (i.e., “endemics”) are remarkably widespread taxonomically, with 246 taxa from 103 genera and 41 families, including angiosperms, gymnosperms, and ferns (Safford et al. 2005), providing the replication necessary to detect evolutionary trends. In the California Floristic Province, serpentine soils have a patchy distribution, typified by island-like rocky outcrops (Harrison et al. 2006). Endemic taxa are often characterized by limited geographic ranges and small population sizes; some 45% of taxa are considered rare or endangered by state and federal agencies (Safford et al. 2005).

The edaphic stress, insular spatial structure, and rarity of endemic plants on serpentine soils lead to several expectations regarding the origins and evolutionary consequences for these habitat specialists. If serpentine outcrops are truly island-like habitats due to their geographic isolation from one another, then their colonists may undergo adaptive radiations leading to increased diversification rates. Alternatively, the environmental homogeneity of serpentine soils combined with the demographic and genetic consequences of the small population sizes characteristic of serpentine endemics could decrease speciation rates and increase extinction rates causing an overall decrease in diversification rates. Another expectation is that the evolutionary transition toward serpentine endemism may be unidirectional. If endemics require a unique combination of physiological adaptations and ecological strategies to specialize on serpentine soils and these adaptations come at a cost of competitive ability that ultimately leads to their exclusion from nonserpentine habitats, then the transition to serpentine endemism may be irreversible.

Here, we examine the origins and consequences of serpentine endemism using a phylogenetic approach. Because it is very plausible that serpentine endemics may have distinct speciation and extinction rates as compared to nonserpentine lineages, and because the methods used to examine irreversibility can be impaired by habitat-dependent diversification rates (Maddison 2006; Goldberg and Igic 2008), we use models that simultaneously estimate habitat-dependent diversification rates and transition rates to and from those habitats (Maddison et al. 2007) in addition to independent analyses of character evolution. We also assess whether endemic lineages are evolutionarily younger than nonendemics, as would be expected if endemics have shorter persistence times associated with high extinction rates—that is, ancient serpentine endemics are more likely to go extinct than neo-endemics. A further constraint on the age of endemic lineages is the relatively recent exposure of serpentine outcrops in some regions of California (Harrison et al. 2004). Collectively, the results provide a new perspective on the evolutionary history of serpentine endemism in the California flora and offer a modern macroevolutionary framework for the investigation of the origin and consequences of other forms of specialization.

Materials and Methods


We used a database of serpentine affinity (Safford et al. 2005) to assign every taxon in the California flora (5800+) to one of three categories of habitat specialization: “serpentine endemic”—taxa with >85% of known occurrences on serpentine soils (score of 4.5 to 6.0); “serpentine tolerator”—taxa observed both on and off serpentine (score of >0 to 4.5); “serpentine nontolerators”—taxa never observed on serpentine soil (score = 0). The scores in the database represent a compilation of observational information on plant affinity for serpentine soils. Sources include a monograph on serpentine endemism in California (Kruckeberg 1984), the Jepson Manual (Hickman 1993), peer reviewed and gray literature, expert opinion, field observation, and herbaria records (Safford et al. 2005). The final database contained 103 genera with at least one serpentine endemic.


We selected 23 genera (784 taxa) from the serpentine affinity database for our phylogenetic analysis of habitat specialization in the California flora (Table S1) based on two criteria: (1) each genus must have at least one taxon endemic to California's serpentine soils; and (2) molecular sequence data must be available for an exhaustive sample of taxa within each genus. Among this sample, some nontolerator taxa have no contemporary range overlap with serpentine soils, and thus may actually be tolerant of serpentine if they had the chance to colonize it. When these taxa were treated as polymorphic (“nontolerator/tolerator”), transition and diversification results (not shown) did not qualitatively differ from those presented below. For the 23 genera in our sample, we first used PhyLoTA [GenBank release:159 (April 15, 2007)] to identify loci with “phylogenetically informative clusters” (Sanderson et al. 2008). Accession numbers for all sequences obtained are listed in Table S3. Due to recent additions to Genbank, some sequence information was not available in earlier versions of PhyLoTA (i.e., Orthocarpus and Trichostema). In such cases, we downloaded sequences directly from GenBank. The raw sequence data were then aligned in BioEdit version 7.0 with ClustalW (Hall 1999) and manually adjusted as necessary. Finally, we sent our habitat specialization classifications for these 23 genera to experts for each genus for further review and incorporated their input as necessary. We include the final character state determinations in Table S3 (Nontolerators, n= 555; Tolerators, n= 178; Endemics, n= 51).

We used the Jepson Manual to determine the total taxon richness for each genus in California and the United States, and calculated the proportion of these taxa that we included in our phylogenetic analysis. We also compared the level of endemism in the genera in our sample (n= 23 genera) with the level of endemism in the genera of the entire California flora (n= 103 genera).

Markov chain Monte Carlo (MCMC) phylogenetic analyses were run in Mr. Bayes (version 3.1.2 [Ronquist and Huelsenbeck 2003]) to obtain a posterior distribution of phylogenetic trees for each genus. Four independent runs of Mr. Bayes were conducted for each genus. Each run consisted of one cold chain and three heated chains that were sampled every 50,000 generations for a total of 10 million generations. Temperatures were adjusted to attain swapping frequencies between 10% and 70% for all chains. For each run, the initial 106“burn-in” trees were removed. We examined model parameters in Tracer version 1.4 and compared the standard deviation of split frequencies between paired runs to confirm convergence and mixing for each run. The posterior distribution of trees for all four runs was then combined to make 720 trees per genus. We used nonparametric rate smoothing, as implemented in TreeEdit (Rambaut and Charleston 2001), to transform all phylogenies in adjustment for the lack of fit of our data to a molecular clock in the majority of our genera (P < 0.05 in 21 of 22 likelihood ratio tests; PAUP4.0b10 [Sinauer Associates, Sunderland, MA]; Aquilegia excluded because of AFLP data). All subsequent phylogenetic comparative analyses were conducted on these ultrametric trees to ensure branch lengths were approximately proportional to time.


To test for directional biases during the evolution of serpentine endemism, we used two approaches to assess transitions independently of diversification: stochastic character mapping and applying constraints to a transition model. First, for each genus individually we used stochastic character mapping on the posterior distribution of trees to infer the average number of transitions between each of the serpentine affinity conditions using 100 realizations per tree for the entire posterior distribution of trees in SIMMAP (Bollback 2006). This approach allows for incorporation of phylogenetic uncertainty and uses branch length information. Because transition results can be affected by the frequency of character states, we compared our results to a series of reshuffled character matrices. To test whether the observed transitions were significantly different from expected, based on randomly distributed character states at the observed frequency, we shuffled the character states 100 times and performed 10 realizations for each tree in the posterior distribution to estimate the average expected number of transitions per genus. We used a chi-square test for each genus to assess whether the number of observed and the expected transitions were significantly different.

The significant bias in the number of inferred transitions motivated an examination of the transition rates between character states. We compared the full model of six transition rates to three constrained models: the rate from endemics to nontolerators (qEN) set equal to zero, the rate from endemics to tolerators (qET) set equal to zero, and the simultaneous restriction of these two rates to zero (Table 1). For each model, we performed MCMC sampling of the rates across the posterior distribution of trees. We then computed Bayes factors (BF) for each model comparison from the harmonic mean of the likelihoods in the MCMC chains. The difference between scores for two models approximates ln(BF) [i.e., ln(BF) = ln(marginal likelihood 1) − ln(marginal likelihood 2)]. Support for one model over another is “strong” if 2 × ln(BF) > 5 (Kass and Raftery 1995).

Table 1.  Summary of diversification and transition analyses used in this study.
AnalysisEstimatesCharacter dataNumber of parameters
  1. Notes: Ternary states are nontolerator (N), tolerator (T), and endemic (E). Binary states are endemic (E) and nonendemic (Ne=N or T). GeoSSE rates are named after habitat types rather than states, S=serpentine, O=other/nonserpentine. Diversification is defined as speciation minus extinction.

Waiting timediversification rates of E and Nebinary2
SIMMAPnumber of transitions between N, T, and E (NT, NE, TE, TN, EN, ET)ternary6
BayesTraitstransition rates (qNT, qTN, qNE, qEN, qTE, qET)ternary6
BiSSErates of speciation (λE, λNe), extinction (μE, μNe), and transitions (qENe, qNeE)binary6
GeoSSErates of speciation (sS, sO), extinction (xS, xO), and dispersal (dS, dO)ternary6

We tested for transition rate biases on individual genera and on all genera simultaneously. First, we used BayesTraits (Pagel and Meade 2006) to analyze each genus separately. This involved the MultiState option, exponential hyperpriors, and a uniform frequency root state assumption; the last assumption can sometimes be problematic (Goldberg and Igic 2008), but that problem renders our test conservative and is mitigated by the presence of a third character state. Second, to increase our power to estimate transition rates, we combined phylogenies for all 23 genera into a joint dataset to allow the estimation of a single set of rate parameters across all genera. This did not mean forming a supertree, which due to the phylogenetic distance between our genera would have suffered from a large degree of incomplete and patchy sampling. Instead, we chose one tree from the posterior set of trees for each genus and formed a joint likelihood function as the product of the individual clade likelihoods; we repeated this many times to obtain a posterior set of tree combinations. The joint likelihood function treats the genera as independent. Although they are not completely independent, they are very distantly related from one another. For instance, the 23 genera sampled represent 17 families. Multiple genera were sampled in only two families (Asteraceae [6] and Apiaceae [2]). Even in these two families, no two sampled genera are sister genera. Furthermore, fixing the root of each genus to nonserpentine to account for the relatively recent emergence of serpentine, as we did for the BiSSE and GeoSSE analyses described below, is a reasonable means to account for the relatively recent emergence of serpentine. Transitions to serpentine affinity character states deeper than the level of genus in our dataset are impossible. When serpentine soils become available, at approximately the age of these genera, transitions to (and from) serpentine then proceeded independently from the nonserpentine root state in each genus, removing the shared history of the genera with regard to this character.

On this set of combined trees, we used tools in the R package diversitree (FitzJohn 2010) to conduct a BayesTraits-like analysis, including estimates of transition rates and model comparisons, as described above for the individual-genus analysis. We first test for rate asymmetry by comparing the full model against two constrained models: qEN=qNE and qET=qTE. Then, we test the full model against each of the three constrained models described above (i.e., qEN= 0, qET= 0, and qEN=qET= 0). In the analysis of the combined phylogenies, we used broad exponential priors and the conditional likelihood root state assumption, which deals naturally with the constraints of irreversible models (FitzJohn et al. 2009).


To allow for character state-specific speciation and extinction rates, we performed two analyses, treating habitat occupancy as a two-state character (BiSSE) or a three-state character (GeoSSE) (Fig. S1; Table 1). The current implementations of BiSSE require hundreds of taxa and are limited to characters with just two states (Maddison et al. 2007; Moore and Donoghue 2009), thus we combined our genera and simplified the character data from three states to two states (serpentine endemics vs. nonendemics). The BiSSE model estimates speciation (λ) and extinction (μ) rates for each character state [endemic (E), nonendemic (Ne)] and transitions to the endemic character state (qNeE) and out of the endemic character state (qENe).

The combination of the tolerator species with nontolerators will, however, obscure the effects of serpentine adaptations on macroevolutionary trends, so it would be more effective to treat the tolerator character state as a combination of the habitat types experienced by nontolerator and endemic lineages. A second model (GeoSSE) was fit to accomplish this. The model was originally described by Goldberg et al. (2005), but it has only recently been applied to phylogenetic data (Goldberg, Lancaster and Ree, in review). Its parameters are tied to soil type (serpentine, S and other, O) rather than to the character states (serpentine endemic, E, tolerator, T and nontolerator, N). The six parameters are speciation rate (sS and sO), extinction or extirpation from a soil type (xS and xO), and dispersal or range expansion to the other soil type (dS from serpentine). Further explanation is given in the Supporting information. Given the state transitions described here, construction of the likelihood function for a clade under the GeoSSE model is a straightforward extension of the procedure for BiSSE (Maddison et al. 2007). Simulation tests (Goldberg, Lancaster and Ree, in review) indicate that parameter estimation is at least as accurate as, and often more precise than, BiSSE. Both models have six rates to be estimated (Table 1), but GeoSSE has the advantage that its three possible character states contain more information than the two possibilities for BiSSE (Fig. S1). For both the BiSSE and GeoSSE analyses, we combined genera as described above to form a single, larger dataset. We also fixed the root to the nonendemic (BiSSE) and nontolerator (GeoSSE) states to reflect the relatively recent appearance of serpentine soils compared to the age of these genera. Analyses were performed with the R package diversitree (FitzJohn 2010).


For each genus, we compared the terminal branch lengths of endemics, tolerators, and nontolerators. First, a majority rule consensus tree was used to summarize the posterior distribution of trees for each genus. We then removed the outgroups and used nonparametric rate smoothing to transform the tree, as implemented in the APE package in R (Paradis 2006). We compared the average terminal branch lengths of endemics, tolerators, and nontolerators per genus. Because of uncertainty in comparing the magnitude of terminal branch lengths among genera due to life-history effects and other factors (Kay et al. 2006), we analyzed endemic versus tolerator and endemic vs. nontolerator average terminal branch lengths for each of the 23 genera using two sample Wilcoxon signed rank tests.



We mined GenBank for phylogenetically significant clusters of taxa and located a total of 19 genera that met our criteria. For many of these 19 lineages, we added taxa not included in previously published phylogenetic analyses by combining samples from different studies. Two additional lineages that appeared in GenBank subsequent to the PhyLoTA release were added to our sample (Orthocarpus and Trichostema). Finally, molecular data for two more lineages were added from previously published phylogenies: Aquilegia (Whittall and Hodges 2007) and Mimulus (Beardsley et al. 2004). The taxa in our genera represented close to exhaustive sampling with respect to California taxa (mean of 99%). In total, our sample for phylogenetic analysis contained 23 lineages and represented the proportion of serpentine endemic taxa per genus found across the entire California flora (Fig. S2). The alignments and Bayesian consensus trees are deposited in TreeBASE. The results from our Bayesian analyses largely corresponded with previously published topologies, and often provided greater resolution at nodes that were only weakly supported in previous studies (Fig. S3).


Given our three habitat states, there are six possible transitions, including two transitions to the endemic state and two transitions out of the endemic state (Table 1). We estimated the number of shifts between habitat states using stochastic mapping across the posterior distribution of phylogenies. Transitions out of the endemic state were the rarest transitions, collectively nearly two times less common than the combined number of transitions that produce endemics (Table S1; Fig. 1). Across the 23 genera, there was a strong bias of transitions that produce endemics and a dearth of transitions out of the endemic state (Table S1; paired Student's t-test: df = 21, P < 0.0001).

Figure 1.

Evolutionary history of serpentine affinity reconstructed with stochastic character state mapping. For four representative genera, we present a single stochastic mapping iteration on one of the 720 smoothed trees from the posterior distribution. Branch colors are black for nontolerators, light gray for tolerators, and dark gray for endemics. Serpentine endemic and tolerator taxa are labeled with colored symbols; unlabelled taxa are nontolerators.

To test for phylogenetic signal in the serpentine endemic character state, we randomized the character states and compared the number of inferred transitions to endemism to the observed data. We found that the observed number of transitions was significantly different than expected in only three of the 23 genera (Aquilegia, Cirsium, and Lessingia). Thus, in most of the genera, the observed transition frequencies do not reflect departures from what would be expected given the frequency of that character state in the lineage (i.e., phylogenetic signal). This lack of phylogenetic signal is consistent with the general rarity of endemics, especially when there is only one endemic per genus.

We next compared the full model of six transition rates to models that prohibit transitions out of the endemic state to test the unidirectionality of serpentine endemism (Table S2). When each genus is considered separately, 82% of the genera (19 of 23) showed no preference for any of the models. The model that omits qET was strongly preferred for one genus (Sidalcea) and strongly unpreferred for one genus (Allium). The model that omits both qEN and qET was strongly unpreferred for three genera (Allium, Aquilegia, and Mimulus) and strongly preferred for one genus (Sidalcea). When the genera were combined into a single analysis, the full model was preferred over models constrained to have symmetric rates to and from endemism (qNE=qEN and qTE=qET; 2 × ln(BF) of 2.2 and 9.7, respectively), meaning that the forward and reverse rates were asymmetric. The posterior distribution from the full model indicates that the weaker preference over the former constrained model is due to low precision in estimating qEN (Fig. 2A). Reversal rates from the endemic state were higher than the forward rates to the endemic state (qEN > qNE with posterior probability of 0.95; qET > qTE also with posterior probability of 0.95) (Fig. 2). The estimates of the reversal rates are quite broad, however, likely because of the relatively small number of endemics. Analyses where reversal rates were set to zero show that the full model was strongly preferred over the constrained models: the 2 × log(BFs) for the comparisons of the full model versus the constrained models (qEN= 0, qET= 0, and qET=qEN= 0) were 13.5, 9.5, and 117.4, respectively. Thus, forward and reversal rates were asymmetric, yet unexpectedly the transition rates out of the endemic state ranged from four to seven times higher than the transition rates into the endemic state (Fig. 2).

Figure 2.

The posterior distribution of transition rates into and out-of the endemic state using an analysis similar to BayesTraits for all 23 genera combined. Bars along the horizontal axis span the 95% credibility interval. In (A) we compare transitions between nontolerators and endemics and in (B) we compare transitions between tolerators and endemics.


Under the simultaneous estimation of habitat-dependent speciation, extinction, and dispersal rates with the BiSSE model, we recover very strong support for a lower diversification rate (speciation minus extinction) for endemics than nonendemics (λE–μE < λNe–μNe with posterior probability of 0.95) (Figs. 3A–C). Serpentine endemics had a lower speciation rate and a lower extinction rate than nonendemics in nearly the entire posterior distribution (λE < λNe with posterior probability of 0.98; μE < μNe with posterior probability of 0.999) (Fig. 3A, B). This model does not find a transition rate bias toward shifts to endemism (qNeE > qENe with a posterior probability of 0.00) (Fig. 3D). In fact, all of the posterior distribution shows qNeE < qENe. The results of the GeoSSE model, which allowed incorporation of the tolerator condition, found serpentine soils to have a lower speciation rate (sS < sO with a posterior probability of 1.0), a higher extinction rate (xS > xO with a posterior probability of 0.99), and a lower diversification rate (sSxS < sOxO with a posterior probability 1.0) (Fig. 4A–C). The diversification rate was approximately 20 times lower for serpentine than nonserpentine habitats. The model also found that dispersal of a serpentine lineage into nonserpentine habitats (dO; thereby creating a tolerator lineage) was nonzero, but the dispersal from a nonserpentine lineage into serpentine habitats (dS; also creating tolerators) was higher in a substantial portion of the posterior distribution (dO > dS with a posterior probability 0.83) (Fig. 4D).

Figure 3.

The posterior distribution of rates of state-dependent speciation (A), extinction (B), diversification (C), and character change (D) using binary data (BiSSE). Bars along the horizontal axis span the 95% credibility interval.

Figure 4.

The posterior distribution of rates for habitat-dependent speciation (A), extinction (B), diversification (C), and dispersal (D) using three-state data (GeoSSE). Bars along the horizontal axis span the 95% credibility interval.


We used the terminal branch lengths extracted from ultrametric trees to compare the relative ages of endemics, tolerators, and nonendemics. In 82% of the genera examined, the average endemic taxa occurred on shorter branches than their tolerator congeners (Wilcoxon signed-ranks, df = 21, P= 0.0016) (Fig. 5A; Fig. S4). In 74% of the genera examined, the average endemic taxa occurred on shorter branches than their nonendemic congeners (Wilcoxon signed-ranks, df = 22, P= 0.0006) (Fig. 5B; Fig. S4). It should be noted that serpentine endemism may have arisen either before speciation, during speciation, or subsequent to the branching event that was used in these relative age comparisons, thereby making endemics possibly older or younger than inferred. Given that the ancestral condition of each genus is perhaps more likely nonendemic than endemic, the comparisons may be biased toward overestimating the age of endemics. However, the potential bias toward over estimating the age of endemics renders our test conservative given the result that endemics are younger than nonendemics.

Figure 5.

Differences in the relative ages of serpentine endemics and serpentine tolerators (A) and nontolerators (B) averaged for each genus. Negative values indicate the mean nonendemic age was greater than the mean endemic age (i.e., endemics were younger). Trifolium not shown for scaling purposes (endemic – tolerator age difference =−0.42; endemic – nontolerator difference =−0.56).


We found that after a plant lineage becomes ecologically specialized to the unique conditions of serpentine soils, subsequent diversification is lower than observed in nonendemic lineages. Although the colonization of insular habitats can lead to adaptive radiations, both BiSSE and GeoSSE showed the transition to specialization typically limits subsequent diversification. Similar results were found for an analysis using the waiting time method (Ree 2005) (Supporting information text: waiting time analysis): 22 of 23 genera exhibited a lower diversification rate along branches inferred to be in the endemic state (except Allium; Fig. S5). The speciation and extinction rates from BiSSE were both low for serpentine endemics, suggesting that the low speciation rate is more likely the cause of limited diversification in serpentine habitats, rather than an increase in extinction rate (Figs. 3A, B).

Possible reasons for the lower diversification rate in endemic lineages may lie in the narrow niche breadth of serpentine habitats and the demographic and genetic consequences of narrow distributions (Kruckeberg 1991; Kay et al. In press). Serpentine outcrops are often referred to as edaphic islands due to their sharp boundaries and patchy distribution. Even though strong diversifying selection may originally lead to ecological speciation, within-patch environmental homogeneity could limit subsequent diversification (Rajakaruna 2004; Baldwin 2005). In addition, the limited adaptive abilities of small, isolated populations with narrow distributions, as typical of many serpentine endemics (Kruckeberg 1991; Rajakaruna and Whitton 2004; Brady et al. 2005) may limit genetic diversity and thus restrict evolutionary potential for subsequent adaptive speciation (Stockwell et al. 2003). Small, isolated populations also face high levels of inbreeding (Mills and Smouse 1994; Crnokrak and Roff 1999) and low probabilities of surviving stochastic environmental and demographic events (Lande 1993). Another potential consequence of small population size and low diversification is a slowing of molecular evolutionary rates (Barraclough and Savolainen 2001), but a comparison of the untransformed branch lengths of 30 sister species pairs shows no evidence of a change in rates of molecular evolution in serpentine endemics (not shown). A comparison of the genetic diversity and effective population sizes of nonendemic and endemic sister pairs would help determine the genetic and demographic mechanisms that limit diversification of serpentine endemics.

Some unsampled genera such as Streptanthus, Hesperolinon, and Eriogonum may exhibit alternative patterns regarding the origin and evolutionary consequences of serpentine endemism. We were unable to include these genera because of insufficient phylogenetic sampling for comparative analyses (M. S. Mayer, pers. comm.; Springer 2006) and low phylogenetic resolution, potentially reflecting their recent and complex evolutionary histories. For example, clades of Streptanthus have endemic taxa with very limited sequence divergence (Mayer and Soltis 1994, 1999; Mayer et al. 1994), consistent with recent radiations. A similar pattern of a recent radiation of serpentine endemics is emerging from Hesperolinon based on cpDNA data (Springer 2006). These two genera have been previously identified as foci of neoendemism (Raven and Axelrod 1978; Safford et al. 2005). The genus Eriogonum also contains a high number of endemics, yet as one of the largest genera endemic to North America (∼250 species), it is proportionately low in serpentine endemics (5.6%) relative to Streptanthus (42.5%) and Hesperolinon (69%). Eriogonum currently lacks an exhaustively sampled molecular phylogeny.

The comparison of speciation and extinction rates across the BiSSE and GeoSSE models reveals an important role of tolerators during habitat specialization. The difference in speciation rate between nonendemics and endemics (mean of 5.3) was greater than the difference between nonserpentine and serpentine soils (mean of 4.3). Higher speciation in serpentine habitats (mean sS= 1.25) but low speciation of serpentine endemics (mean λE= 0.29) suggests that endemics are often derived from speciation within serpentine populations of tolerator ancestors rather than from within-serpentine splitting of already-endemic lineages. Similarly, the rate of extinction from serpentine habitats is higher than the rate of global extinction of endemics (xS≫μE). This indicates a relatively high frequency of serpentine populations of tolerator species that go extinct and result in nontolerator lineages.

A possible pathway that would lead to the isolation of small tolerator populations on serpentine outcrops is partial range depletion, in which environmental change drastically reduces the range of the tolerator ancestor but does not drive it completely extinct. Some of the remaining populations may be of small geographic extent and thus likely confined to a single habitat type; divergence due to mutation and drift of the isolated population could then follow and lead to reproductive isolation. This scenario is consistent with evidence from Streptanthus glandulosus complex (Mayer et al. 1994). A second scenario of complete range depletion (extinction of all nonserpentine populations) is also supported by our data, but to a lesser degree (xO is nonzero, and slightly greater than μNe). The loss of all nonserpentine populations would cause a change in character state from tolerator to endemic, but no change in tree shape (i.e., no extinction, no speciation), as opposed to the first scenario in which speciation adds a new node to the phylogeny. Clearly, the exact pathways to both reproductive isolation and serpentine restriction in any particular lineage can only be fully understood through detailed examinations of individual lineages.

Our analysis of transitions among nontolerators, tolerators, and endemics does not support the hypothesis of unidirectional movement to serpentine soils. Stochastic character mapping found significantly fewer transitions out of the endemic state than into it, but the test of phylogenetic signal indicates that the limited reversals are caused simply by the rarity of the endemic state that is associated with low diversification described above; with few endemic species available, few transitions out of the endemic state will be possible, regardless of the ease of such a transition. Furthermore, many of the individual genera are small and therefore lack power (i.e., low number of transitions or total tree length) to test for directionality and irreversibility, as evidenced by the inconclusive results of the BayesTraits model comparisons of irreversible transition rates on a per-genus basis. Analyzing the genera jointly, however, showed a strong preference for the fully reversible model over the irreversible models. In fact, the transition rates to and away from serpentine endemism are asymmetrical, but the transition rates away from endemism were actually higher than the transition rates to endemism (Figs. 2 and 3). Thus, endemics appear to expand easily out of serpentine (high reversal rate). However, extinction levels on nonserpentine soils were high, so reversals are countered by extirpation of the nonserpentine populations.

The reversal from endemism only requires that endemic species have the ability to grow on more fertile soils in the absence of competitors. When competitors are removed from nonserpentine habitats, experimentally or by disturbance, endemics easily expand out of serpentine. However, they are likely extirpated when superior competitors return (Gulmon 1992; Hooper and Vitousek 1997; Liancourt et al. 2005; Elmendorf and Moore 2007). Transitions toward endemism, on the other hand, require the evolution of stress tolerance traits, or the loss of all nonserpentine populations. The difficulty of gaining the adaptations for heavy metals, low fertility, and drought of serpentine is evidenced, in part, by the 77% (4447 of 5800) of the taxa in California that are not known from serpentine. Thus, the multifaceted challenges that plants must overcome to persist on serpentine limits their origins, while their ability to grow on nonserpentine when competition is low may explain the unexpectedly high reversal rates out of the endemic state. However, despite a high reversal rate, reversals are infrequently observed because of the rarity of the endemic state and because reversals from endemism ultimately yield to local extinction.

The results that endemics were younger than tolerators in 82% of the genera surveyed, and younger than nontolerators in 74% of genera, are consistent with short lineage persistence times in serpentine habitats (Fig. 5). Given that we used an analysis of waiting times in a phylogenetic framework to assess diversification, we can rule out the possibility that limited diversification of endemics is due solely to their relatively young age. Because there was no detectable difference in rates of molecular evolution associated with serpentine endemics, we can use a life-history corrected average rate of internal transcribed spacer (ITS) substitution (Kay et al. 2006) to compare the approximate ages of serpentine endemics with important time points in the geologic and climatic history of California. Based on the average ITS substitution rates, the majority of endemic taxa (77%) are younger than the oldest serpentine outcrops in California (50 million years ago [mya]) and more than half of the endemic taxa (54%) are younger than the onset of the Mediterranean climate in California (15 mya) (Harrison et al. 2004).


Extending this analysis of habitat specialization to other serpentine floras around the world will allow us to test if similar patterns underlie serpentine biodiversity everywhere. For instance, it would be very useful to test for similar macroevolutionary trends in the floras of Cuba, where 33% of the genera endemic to the island are endemic to serpentine (Borhidi 1991, 2001); New Caledonia, where serpentine endemism may actually be the ancestral state in many lineages (De Kok 2002); and Turkey, where endemism appears to increase rather than decrease diversification (e.g., Alyssum) (Brooks 1987). The most limiting factor will be the availability of phylogenetic information for plant clades outside of North America and Europe (Anacker, in press).

The extension of this approach to investigations of the origins and consequences of other specialists in the California flora (i.e., vernal pool endemics, hummingbird-pollinated plants, etc.) would add generality to our finding of decreased diversification following the origin of habitat specialization and help reveal new macroevolutionary patterns in this biodiversity hot spot. For example, in Navarretia, serpentine endemism appears to have repeatedly resulted in an evolutionary dead-end whereas vernal pool endemism may have spurred a recent radiation (Spencer and Porter 1997). It would also be ideal to create systematic accounts of affinity for soil types other than serpentine in California and in other serpentine-endemic rich regions, but progress in this regard will be difficult due to the lack of studies of edaphic endemics outside the serpentine system. Ideally, comparative analyses such as ours will continue to generate hypotheses testable with manipulative experiments at the population level, ultimately revealing the genetic and ecological parameters responsible for biodiversity generation and maintenance.

Associate Editor: J. Vamosi


We thank B. Baldwin and A. Hipp for insightful conversations on methods; J. Baty and P. March for technical support; H. Safford and J.H. Viers for help with the serpentine affinity database; B. Baldwin, P. Fiedler, S.R. Hill, L. Johnson, N. Kramer, S. Markos, D. McNeal, M.S. Park, T. Parker, L.E. Urbatsch, D. Wilken, and C. Wilson for expert opinion on serpentine affinity determinations; and B. Moore and C. Dick who commented on a previous draft of the manuscript. This work was partly supported by an Achievements Rewards for College Scientists Foundation scholar award (to BLA), a Comparative Biology Postdoctoral Fellowship in the Section of Evolution and Ecology at UC Davis (to JBW), IPY#0733078 (to JBW), and NSF DEB-0919089 (for EEG).