The study of modularity can provide a foundation for integrating development into studies of phenotypic evolution. The dentition is an ideal phenotype for this as it is developmentally relatively simple, adaptively highly significant, and evolutionarily tractable through the fossil record. Here, we use phenotypic variation in the dentition to test a hypothesis about genetic modularity. Quantitative genetic analysis of size variation in the baboon dentition indicates a genetic modular framework corresponding to tooth type categories. We analyzed covariation within the dentitions of six species of Old World monkeys (OWMs) to assess the macroevolutionary extent of this framework: first by estimating variance–covariance matrices of linear tooth size, and second by performing a geometric morphometric (GM) analysis of tooth row shape. For both size and shape, we observe across OWMs a framework of anterior and postcanine modules, as well as submodularity between the molars and premolars. Our results of modularity by tooth type suggest that adult variation in the OWM dentition is influenced by early developmental processes such as odontogenesis and jaw patterning. This study presents a comparison of genotypic modules to phenotypic modules, which can be used to better understand their action across evolutionary time scales.

Since Darwin, evolution has been characterized as change over time in population-level phenotypic variation that ultimately results in species differences. Although the Modern Synthesis gave us an analytical framework for understanding that allelic variation at the population level can underlie macroevolutionary phenomena (e.g., Simpson 1944), a similar theoretical framework is needed to understand the translation of heritable information into phenotypes upon which natural selection can act. Our increasing understanding of developmental genetics and the complexity of phenotypes require that we incorporate a more detailed treatment of the ontogenetic development of phenotypes into current evolutionary theory. Indeed, some have even called for an Extended Evolutionary Synthesis to do exactly this (Pigliucci 2007).

Although still in its early stages as a formal theoretical framework, the study of morphological integration and modularity has the opportunity to provide the foundation for integrating development into studies of phenotypic evolution. From the early work of Olson and Miller (1958), the similar idea of correlation pleiades (Berg 1960), and the elaboration of these ideas by Gould and Garwood (1969), Cheverud (1982, 1995, 1996a), and Wagner (1996), this subdiscipline of evolutionary biology has begun to interpret developmental effects from phenotypic variation (see review in Willmore et al. 2007).

Morphological integration is the idea that some aspects of the phenotype will covary based on shared pleiotropy, development, function, and/or selective pressure. The study of modularity refers specifically to the identification of these integrated morphological units, or modules, which are typically classified as genetic (G matrix-defined), developmental (defined by mechanistically correlated precursors of a trait), functional (defined by a trait's interaction with other body parts performing a particular function), or evolutionary in character (defined by observation of correlated change over time) (Klingenberg 2008). These modules are not mutually exclusive and all influence each other in turn, with phenotypic covariation being the product of all of these modules (Cheverud 1996a).

The first step to actualizing this theoretical framework is to acquire more data on the behavior of modules in different phenotypic traits and different evolutionary contexts. Many investigations of modularity are attempting to map genetic or developmental modules onto the patterns of phenotypic variation and covariation seen at the population level or higher (e.g., Klingenberg and Zaklan 2000; Hallgrímson et al. 2002; Willmore et al. 2006). However, the functional and developmental complexity of traits such as the mammalian cranium or mandible (which comprise the majority of studies to date) make it difficult to identify genetic modules beyond the division between the face and neurocranium (Cheverud 1982, 1995; Halgrímsson et al. 2004; Goswami 2006; Drake and Klingenberg 2010) or the alveolar and ascending ramus modules recognized by Klingenberg et al. (2004; see also Zelditch et al. 2008; Perez et al. 2009) (but see Marquez 2008and Parsons et al. 2012 for recent methodological advances).

In contrast, the relatively simple serial homology of the tooth row provides a more tractable phenotype for identifying traits that compose/encompass genetic modules. Although studies of molar cusp morphology (Polly 2005) and shape within the molar row (Miller et al. 2007; Laffont et al. 2009) are applicable, given the evidence for early fate determination of tooth type based on position along the tooth row (reviewed in Jernvall and Thesleff 2000; Salazar-Ciudad 2008; Cobourne and Sharpe 2010), we argue that studying the relationships between teeth within the whole tooth row may be the most informative approach to identifying patterns of dental modularity. Additionally, while most skeletal features change over an individual's lifetime in response to bone remodeling and repair, mammalian teeth acquire most of their structure prior to birth (except for wear) thus reducing the nonheritable effects on adult phenotypic variation. In terms of modularity, this means that genetic and developmental modules map very closely to each other. We suggest that the dentition may therefore represent an ideal phenotype for studying the genotype–phenotype map as it is developmentally relatively more simple, adaptively highly significant, and evolutionarily tractable as their largely inorganic content makes them common in the fossil record.

Hlusko et al. have undertaken a quantitative genetic analysis of mammalian dental variation (Hlusko and Mahaney 2003, 2007a; Hlusko et al. 2004a,b; Rizk et al. 2008). Their studies of baboon dental variation reveal patterns of pleiotropic effects within and between the various teeth and across the maxilla and mandible (Hlusko et al. 2002, 2006, 2007, 2011; Hlusko and Mahaney 2007b,a; Koh et al. 2010). This work provides evidence for a genetic modular framework underlying baboon tooth size variation that corresponds to tooth type categories: an incisor module that is independent of the postcanine dentition, and a postcanine module with premolar/molar submodules (the premolars and molars are influenced by incomplete pleiotropy). The next step is to determine whether this modular framework characterizes dental phenotypic variation in other Old World monkeys (OWMs).

The OWMs (Primates:Cercopithecidae) diverged from a common ancestor 25 million years ago (Zalmout et al. 2010) and have since diversified, migrated across Africa and Asia, and now exploit a fairly broad range of habitats, from terrestrial herbivores to arboreal folivores. The OWM dentition is representative of a generalized mammalian dentition in being diphyodont and heterodont with an adult dental pattern of two incisors, one canine, two premolars, and three molars per quadrant. Serendipitously, they have also been incorporated into vertebrate skeletal collections at numerous museums around the world, and therefore large sample sizes are available for extensive phenotypic studies.

Here, we present results from two analyses that test the hypothesis that the genetic modules noted in previous quantitative genetic analyses characterize OWMs more generally. We estimated variance–covariance matrices of linear tooth size for six species of OWM from two subfamilies to test whether the tooth type modular framework identified in baboons extends across OWMs, looking for evidence of this modular framework in species variation (microevolution) and across the group as a whole (macroevolution).

To examine the extent of this modular influence on other tooth-related phenotypes, we performed a geometric morphometric (GM) analysis of tooth row shape variation to see whether size-corrected shape variation reflects the same modular framework by analyzing the dentition without the constraints of traditional anatomical traits. The prediction for the shape data under the genetic modular framework is that tooth types will vary categorically as defined by the G matrix for baboon tooth size. This latter analysis provides empirical data for the behavior of a modular framework across micro- and macroevolutionary levels. Together, these studies connect genotypic modules to phenotypic modules, which can be used to better understand their influence across evolutionary timescales.


Measurements were collected from 608 skeletonized crania from four museums: American Museum of Natural History (New York), Cleveland Museum of Natural History (Ohio), National Museum of Natural History (Smithsonian Institution), and the Museum of Vertebrate Zoology (University of California Berkeley). These crania represent the two extant subfamilies (Colobinae and Cercopithecinae) of OWM, following the taxonomy of Xing et al. (2005) and Meyer et al. (2011). The Colobinae in this study consist of the African Colobus guereza and the Asian Presbytis melalophos and Pr. rubicunda. The Cercopithecinae consist of three genera, two from Africa (Cercopithecus mitis and Papio hamadryas) and one from Southeast Asia (Macaca fascicularis) (Fig. 1). The ratio of males to females within most samples is near 1:1, however M. fascicularis and Pa. hamadryas are exceptions in which the ratio is skewed toward males (Fig. 4). To control for ontogenetic variation, we restricted our study to adult monkeys whose dentitions included fully erupted third molars. See Table 1 for summary and Table S1 for full list of specimens.

Figure 1.

OWM phylogeny with estimated times of divergence. Branch lengths not drawn to scale. Taxonomic relationships and divergence dates after Xing et al. (2005) and Meyer et al. (2011).

Figure 4.

Within-species first principal components of shape variation. Shape deformation along the first principal axis of variation within each species (yellow polygons) is represented as change (arrows) from each mean species shape (gray polygons). Note that the line segments connecting select landmarks to create the polygon shapes depicted here do not represent actual shape variation detected by the GM analyses, but function only as an aid for visualizing the changes in individual tooth shape. Deformations were not qualitatively altered after standardizing for CS, but the proportion of variance explained is reported for unstandardized shape data.

Table 1.  Specimen summary.
TaxonGenus and species M F U AMNHCMNHMVZNMNH N
  1. M, male; F, female; U, sex uncertain; AMNH, American Museum of Natural History; CMNH, Cleveland Museum of Natural History; MVZ, Museum of Vertebrate Zoology, University of California Berkeley; NMNH, National Museum of Natural History.

Cercopithecinae Cercopithecus mitis 4544655213795
  Macaca fascicularis 59 29 10 86 5 7 0 98
  Papio hamadryas 6336284274155127
Colobinae Colobus guereza 59 48 18 53 4 0 68 125
  Presbytis melalophos 4041258002583
  Presbytis rubicunda 38 41 1 36 0 0 44 80
 Study Totals =304239652923849229608

Landmark data were collected from a subset (n= 315) of the 608 specimens noted above. Monkeys missing bilateral pairs of landmarks due to missing, chipped, or excessively worn teeth were not included in our sample due to landmark collection constraints (see below). Monkeys for which sex could not be identified were also excluded from landmark analyses.



Photographs were taken of each specimen's maxilla using a Nikon D80 camera with a Nikkor AF-S 105-mm micro lens (Nikon Corp., Melville, NY). The camera was set on manual focus, AV f32, ISO 200, and white balance incandescent. Images captured are 300 dpi and approximately 33 × 21.5 cm. All specimens were oriented with the postcanine occlusal surface in the focal plane. These photographs were used to measure molar lengths and widths and to collect landmark data. Error introduced by photography was assessed to be below 2% and likely reflects inter- and intra-observer measurement error more so than error introduced by the photography protocol (repeated photographs of the same specimen were measured and differences between the measurements were calculated).

Ten linear measurements of the maxillary incisors, canines, and premolars were collected by hand using calipers, and follow common practice in primate odontometry (Swindler 2002; see Fig. 3 for explanation of measurements). Linear measurements (n= 9) for the molars mimicked standard measurements of these teeth as if they were taken by caliper, but were taken using ImagePro Plus (Media Cybernetics, Inc., Bethesda, MD). All caliper data were collected by one observer (OTR). Six research assistants collected the linear measurements from photographs; each person collected all data from one species. Intra-observer measurement error averaged 1.3%.

Figure 3.

Phenotypic correlation matrices for tooth size traits. The strength of correlation is categorized by color, such that the higher correlations are in yellow and the lower correlations in blue. Correlations for which zero is included in the 95% confidence interval are indicated by a slash across the number. Abbreviations: I, incisor; P, premolar; M, molar; number following first letter indicates tooth position; LL, labiolingual width of the incisor; MD, mesiodistal length of the incisor; L, mesiodistal length (the longest mesiodistal axis of the premolar or molar), W, maximum buccolingual width of the premolar (not necessarily perpendicular to the mesiodistal length); AW, maximum buccolingual width of the tooth, across the mesial-most pair of cusps on the molar (not necessarily perpendicular to the mesiodistal length); PW, maximum buccolingual width of the molar through the distal cusp pair (not necessarily perpendicular to the mesiodistal length).

Two-dimensional landmark data were digitized from the photographs using tpsDig 2.10 (Rohlf 2006). A total of 93 landmarks were selected to represent overall dental arch shape, as well as the shape of individual teeth within the arch, following the criteria set forth by Zelditch et al. (2004). Landmarks are defined in Table S2 and illustrated in Figure 2. Bilateral landmarks (all landmarks except the midline incisor landmark 47) were reflected across the midline and averaged using the program BigFix6 (Sheets 2001a), reducing the total number of landmarks used for analyses to 47.

Figure 2.

Landmarks used in the geometric morphometric analysis. Ninety-three two-dimensional landmarks taken from photographs of Old World monkey maxillary dentitions (see Table S2 for landmark definitions).

All landmark data for a given taxon were collected by the same observer. We measured error due to landmark identification using multivariate analyses of variance around each landmark (Corner et al. 1992). The average SD for landmark location for four specimens with landmarks collected 10 times each was 0.125 mm. The range of SDs for landmark precision was 0.0196–2.67 mm, where the largest values were from trials in which nearby landmarks were collected in reverse order. Systematic patterns in error, interobserver precision, and accuracy were addressed verbally among the four observers prior to collecting the landmark data.


Descriptive statistics for the linear measurements reported in Table S3 were estimated using JMP 9.0.2 (SAS Institute, Inc., Cary, NC). Correlations reported in Figure 3 and Table S4 were estimated between all possible pairwise trait comparisons for individuals within a species using the pairwise method in JMP 9.0.2 (SAS Institute, Inc.).


Shape variation within each species was assessed by performing a Procrustes superimposition of landmark configurations for each taxon using Coordgen6 (Sheets 2001b). A principal components analysis (PCA) of shape variation was performed and centroid sizes (CSs) were calculated for each dataset as described in Grieco and Rizk (2010). CS is the measure of size used for scaling in a Procrustes superimposition for its independence from shape in the absence of allometry (Zelditch et al. 2004). Males and females were distinguished in PCA plots as a preliminary assessment of sexual dimorphism. Outliers were identified visually and removed to assess their impact on the analyses. Each taxon was also separately standardized with respect to CS and to sex in Standard6 (Sheets 2001c) and compared to results on unstandardized coordinates. Standardization did not substantially change the observed axes of shape variation. We report here the unstandardized within-species variation as shape deformations from the mean landmark configuration.

The effects of allometry and sexual dimorphism on shape were also tested for at the family level. A Procrustes superimposition of all specimen configurations was performed to obtain an aggregate shape dataset, which was then subjected to a multivariate analysis of covariance with sex and species as factors, and with CS as a covariate. Because this test was performed on the aggregate dataset of 315 specimens, it was not underpowered and could yield interpretable results using a permutation-based approach. This method differs from a traditional MANCOVA in that it does not compare ratios of variance, but instead compares partial Procrustes distances within and between designated groups (12 in our model) to a permuted dataset (Zelditch et al. 2004). It uses explained sum of squares (SS) as the test criterion, and the percent variance explained by the model is calculated as the explained SS divided by the total SS. The covariate-only model allows for common slope and different means, whereas the full model allows for different slopes and means for each term. Tests were conducted in the program Manovaboard6 with 100 permutations (Sheets 2006).

A covariate-only model was compared to the full model that includes sex by CS and species by CS interaction terms to test whether categorical sex and species variables are redundant with the CS covariate. Randomizing the values for either sex or species in the full model returns a statistically significantly worse fit than the full model (P < 0.0100 for each test), so both appear to be “good” factors.

Because we view sex, species, and CS as biologically relevant factors and our aim is to interpret influences on shape variation, we use a more complex model to account for the shape variance in our sample despite the statistically better fit of the covariate-only model (P= 1.000 that the more complex model is a better fit). As shown by comparing the covariate-only model to the full model (59.62% explained, P < 0.01 and 61.81% explained, P < 0.01, respectively, see Table 2), most of the significant effects of sex and of species on shape are additive with CS, but there is an interaction between sex and CS (P < 0.0100). Taken together, we favor a model that includes the sex by CS interaction along with sex, species, and CS in our model as explanatory variables.

Table 2.  MANCOVA on aggregate shape data.
Covariate only versus full model
ModelTotal SSExplained SSPercent variance explained P-value
  1. CS, centroid size; SS, sum of squares; P-value after 100 bootstrap permutations.

CS only1.42990.852559.62<0.0100
Sex×species×CS 1.4299 0.8838 61.81 <0.0100
MANOVA with interaction term    
 CS, before standardization1.42990.353924.75<0.0100
 Sex, species, sex×species, after standardization 1.076 0.4929 45.8 <0.0100

To partition variance among sex and species, a two-way MANOVA was performed after standardization via the CS covariate. The covariate explained 24.7% of shape variation (P < 0.0100) in a model that included the sex by CS interaction. After standardizing for CS, the two factor plus single interaction model explained 45.8% of the remaining variance in shape (P < 0.0100, Table 2). This model performed better than either model including a single factor and its interaction term with the covariate (data not shown).

The aggregate shape dataset was sequentially standardized with respect to CS and then sex in Standard6, thereby extracting the shape residuals from regressions on CS and then sex for further analysis (Sheets 2001c). To visualize the effects of sex and size on shape in the dataset, we performed a PCA and compared the distribution of individuals along the PC1 and PC2 axes for the aggregate shape variables before and after standardization. As with individual species datasets, standardization did not markedly change the observed shape variation in the aggregate dataset, but it decreased the proportion of variation attributed to each PC, as expected from our comparison of the full model to the covariate-only MANCOVA (see Figs. S1, S2).


Placement of all species within a common morphospace allows a quantitative comparison of shape variation between each species. Parsons et al. (2009) caution that common morphospace analyses may be biased by the group with the greatest amount of variance and suggest that such comparisons are only appropriate in cases where group sample sizes are equal and the direction of covariance is consistent across groups. Therefore, a preliminary pairwise comparison of unstandardized species shape variances was performed to test whether they occupy the same morphospace. Following Zelditch et al. (2006), the program SpaceAngle (Sheets 2002) was used to determine the minimum angle of rotation required to align the three-dimensional subspaces of each pair of species (i.e., based on the morphospace spanned by the first three PC axes). The statistical significance of the angle for each pairwise comparison was evaluated using 95th percentile values obtained by 900 bootstrap permutations of resampling within species. Nonsignificant angles for every pairwise comparison (see Table S5) confirmed the consistency of covariance required for a common morphospace analysis to be biologically meaningful.

The aggregate shape variation residuals following sequential regression on size and sex were examined for the similarity of within-species variational trajectories. CS alone explains 24.86% of the shape variance and sex explains 8.44% of the variance after standardization for CS. In total, the two factors explain 36.09% of the shape variance in the common morphospace. Although standardization removes a substantial portion of the biology in question, examining the remaining shape variation has the potential to reveal species-specific and/or local patterning effects on modularity and shape variation.

We assessed the similarity of the shape variation specific to each species within this common, standardized morphospace in two ways. First, we performed an additional set of subspace comparsions with SpaceAngle, using each specimen's partial warp scores from the superimposition of the aggregate dataset to compare the major axis of variation for each species. The subspaces compared in this analysis were defined by a single axis, hereafter referred to as a phenotypic vector, for which significant differences were tested using 95th percentile values obtained by 900 bootstrap permutations of resampling within species. This approach derives phenotypic vectors from the entire shape-space occupied by each species.

Second, for each species, we extracted each individual's PC1 and PC2 scores from a PCA within the common morphospace. Together, PC1 and PC2 account for 67.2% of shape variation across the family. We then defined the major axis of phenotypic variation for each species by performing a second PCA on the extracted PC1 and PC2 scores. The phenotypic vector acquired in this way corresponds to the PC1 eigenvector obtained in the second PCA and specifically quantifies the similarity observed across species in the plot of the standardized aggregate dataset along PCs 1 and 2 (Fig. S2). We assessed the overall similarity of these phenotypic vectors by directly comparing the angles between the phenotypic vectors of each species in a pairwise fashion. The statistical significance of the angle for each pairwise comparison was evaluated using 95% confidence intervals (CIs) obtained by 1000 bootstrap permutations in which taxon assignment was randomized. The second PCA and bootstrapping analyses were performed in R (R Core Development Team 2010) using the “boot” package (Davison and Hinkley 1997; Canty and Ripley 2010) (see Supporting information for R script).



Tooth size variation

Correlations for all pairwise comparisons of the 19 linear measurements of tooth size are presented in Figure 3 (see Table S3 for univariate statistics of all measurements; Table S4 reports the 95% CIs for the correlations shown in Fig. 3). These six correlation matrices follow the same overall pattern seen in the G matrix reported for a pedigreed population of baboons (Hlusko and Mahaney 2009a), where the highest correlations are found within tooth classes, an intermediate level of correlation between premolars and molars, and the lowest level of correlation between incisors and the postcanine teeth.

Tooth row shape variation

Within-species PCA deformations. 

The first two principal components of unstandardized OWM shape data combine to explain much of the total variation in dental arch shape in each taxon, ranging from 38.8% explained in Pr. rubicunda to 63.7% explained in M. fascicularis (Figs. 4,5). Across all taxa examined, the shape deformations with respect to each taxon's mean configuration corresponding to PCs 1 and 2 are remarkably similar. The shape deformation associated with the first principal component is a labial translation of the incisors paired with a mesio-distal contraction of the postcanine teeth (Fig. 4). There is also moderate, although nonuniform, change in the canine landmarks (see below). The deformation associated with PC2 includes buccal translation of the premolars and clockwise rotation of the entire molar row, in concert with variable alteration of the canine and incisors (Fig. 5). We observe this pattern across PC2 for all taxa except Pr. rubicunda, which shows buccal translation of the premolars, but more mesio-distal expansion of the molar row and extreme shape changes in the canines and incisors (see below).

Figure 5.

Within-species second principal components of shape variation. Shape deformation along the second principal axis of variation within each species (blue polygons) is represented as change (arrows) from each mean species shape (gray polygons). Note that the line segments connecting select landmarks to create the polygon shapes depicted here do not represent actual shape variation detected by the GM analyses, but function only as an aid for visualizing the changes in individual tooth shape. Deformations were not qualitatively altered after standardizing for CS, but the proportion of variance explained is reported for unstandardized shape data.

For all but one of the species included in this study, when sex and CS are removed from the raw data there is very little change in the shape deformation representing first and second principal components, although the proportion of total variation explained by them decreases. This suggests that size and sexual dimorphism vary along the same phenotypic trajectory as does the residual shape variation. Based on individual species PC plots, tooth row shape differentiation by sex varies by species from distinct clustering of the sexes in Pa. hamadryas and Ce. mitis to no separation at all in the case of Co. guereza (data not shown). The axis best separating the sexes is often PC1, except for Pr. rubicunda.

As noted, the exception to the trend is Pr. rubicunda. For this taxon, when sex is subtracted from the raw shape data, the second principal component follows a somewhat different trajectory (Fig. S3). This indicates that, unlike the other monkeys, the variance due to sexual dimorphism is either a larger component to the overall variance, or that the morphology attributable to sexual dimorphism varies in a different direction. All-taxon plots of PC1 and PC2 that include sex or that remove sex clearly show that sex influences the phenotypic trajectories differently in Pr. rubicunda compared to the other five OWM species (Figs. S1, S2).

Morphospace occupation. 

The PC1 and PC2 shape deformations for each taxon individually are qualitatively quite similar. To quantitatively assess this similarity, we compared the shape deformations corresponding to each taxon's variation within the common size- and sex-standardized morphospace containing all six species (Figs. S1, S2 for plots). Within this common morphospace, we performed a pairwise comparison of the angle between phenotypic vectors representing the main axis of variation for each taxon derived from both the partial warp scores of the aggregate superimposition (performed using SpaceAngle) and the PC1 and PC2 scores from PCA of the aggregate superimposition (performed in R) (Table 3). In cases where the angle between the two vectors was outside the 95% CI, we rejected the null hypothesis that the two vectors were the same (Tables S6 and S7 for CIs). For the majority of pairwise comparisons, we could not reject this hypothesis, in accord with the overall qualitative similarity of the PC1 and PC2 shape deformations across all taxa.

Table 3.  Common morphospace comparison of sex- and size-standardized species phenotypic vectors.
  Cercopithecus mitis Colobus guereza Macaca fascicularis Papio hamadryas Presbytis melalophos Presbytis rubicunda
  1. Top subtable reports the percentage of standardized within-species PC1/PC2 variance explained by phenotypic vectors calculated in the R analysis. Bottom subtable reports pairwise comparisons of phenotypic vectors. The first entry is the acute angle between phenotypic vectors for the indicated taxon pair, measured in degrees, as calculated by SpaceAngle (using complete shape-space information). Bold indicates that the value exceeds the 95th percentile value for 900 independently bootstrapped samples within both taxa (Table S6). The second, parenthetical entry is the acute angle between phenotypic vectors for the indicated taxon pair, measured in degrees, as calculated in R (using only the shape-space described by PC1 and PC2 scores). Bold indicates that the value exceeds the 95% confidence interval for 1000 bootstrapped samples of the paired taxa (Table S7).

Percent σ2 explained87.561.763.768.684.287.3
Cercopithecus mitis   40.3 (9.6) 19.3 (3.3) 32.7 (61.1) 37.0 (5.5) 28.2 (21.4)
Colobus guereza   30.0 (12.9) 66.3 (70.7)27.8 (4.1) 33.6 (30.9)
Macaca fascicularis     41.6 (57.8) 32.6 (8.8) 25.5 (18.0)
Papio hamadryas     59.0 (66.6)46.0 (39.8)
Presbytis melalophos       36.4 (26.9)
Presbytis rubicunda       

Genera fall along a continuum that is a linear combination of the aggregate, unstandardized PC1 and PC2 axes (Fig. S1), demonstrating that at a macroevolutionary level, the OWMs have diversified morphologically along an axis rotated from the within-species phenotypic vectors. Sexual dimorphism, as in individual species plots, splits the sexes along a vector more in line with the within-species phenotypic vectors, except within Pr. rubicunda.


Tooth size variation

Although the phenotypic correlation matrices all demonstrate the same basic pattern suggestive of anterior and postcanine modules, more detailed consideration also indicates some differences between the taxa (Fig. 3). Two differences potentially differentiate the Asian colobines (Pr. melalophos, Pr. rubicunda) from the African colobine (Co. guereza) and cercopithecines (Ce. mitis, M. fasicularis, Pa. hamadryas). First, the two Asian colobine species have lower correlations between the length and width measurements of their canines (rho = 0.36 and 0.40, respectively) compared to the cercopthecines (rho = 0.80, 0.89, and 0.95, respectively) and the African colobine (rho = 0.76). Second, the correlations between premolar and molar size are lower in the two species of Presbytis compared to the other taxa. Additionally, canine size in Pa. hamadryas is highly correlated with molar size variation, a pattern not seen in any of the other five taxa.

Tooth row shape variation

Within the similar-looking PC1 and PC2 shape deformations there exists considerable species-specific variation in the canines. The relative position of canine landmarks in the PC1 deformation varies based on the jaw curvature in a given taxon's mean landmark configuration. In the colobines (Pr. melalophos, Pr. rubicunda, and Co. guereza), the taxa whose mean configurations show the most curved jaw shapes, the shape change associated with the canine in PC1 is a clockwise rotation such that the mesial and lingual landmarks are shifted buccally whereas the distal landmark is shifted lingually in concert with the changes described above (Fig. 4). The cercopithecines with their more parallel postcanine tooth rows are distinct from this colobine trend. In Ce. mitis, the canine is elongated along the mesio-distal axis and slightly rotated clockwise, and M. fascicularis also has a slight clockwise rotation (Fig. 4). In Pa. hamadryas, the taxon with the straightest jaw, the canine is not rotated but instead is elongated along the mesio-distal tooth row axis (Fig. 4).

There is a considerable amount of between-taxon variability in the canine landmarks associated with the buccal displacement of the premolars and rotation of the molar row in PC2. In P. hamadryas, the associated canine shape change is a lingual displacement of the distal canine landmark (Fig. 5). Cercopithecus mitis and M. fascicularis are similar to each other in having a counter-clockwise canine rotation, whereas in Co. guereza and Pr. melalophos, the canine shows very little shape change (Fig. 5). The shape change in Pr. rubicunda differs from these taxa by having a clockwise-rotating canine (Fig. 5). Neither of these species-specific canine differences correlates directly with a taxon's body size dimorphism ratio (data not shown).

Similar to the canines, the displacement of landmarks in the incisor region is consistently dramatic, and the direction of displacement along PC2 varies by species. The deformation associated with the described postcanine variation in Co. guereza, M. fascicularis, and Pr. melalophos is mesial translation of the incisors. Cercopithecus mitis adds to this translation a concomitant expansion of the incisors. In Pa. hamadryas, the incisors contract labio-lingually rather than translating relative to the rest of the tooth row. Presbytis rubicunda continues its unique morphology along PC2 with a labio-lingual expansion of the incisors.


We analyzed covariation within the dentitions of six OWM species using two different approaches: the first estimated correlation matrices of linear size measurements, and the second employed GM. These two different datasets and analytical approaches primarily captured covariation in size and covariation in shape, respectively. In both datasets, we detected similar patterns of phenotypic covariance, which are pervasive across these OWM dentitions despite significant variation in the degree of sexual dimorphism, diet, social structure, a wide geographic range, and 25 million years of evolutionary divergence. Moreover, we find that these patterns of covariation are consistent with modular hypotheses suggested by quantitative genetic data for a model OWM taxon. Together, these findings argue for a genetically based modularity in dental arcade variation in OWMs.

P correlation matrices of linear measurements suggest that the modular framework of anterior and postcanine dentitions reported in baboons (Hlusko and Mahaney 2009b; Hlusko et al. 2011) characterizes OWMs more broadly. In each species, correlations are higher among teeth within a tooth class and lower between tooth classes, with somewhat elevated correlations between molar and premolar classes. These are similar to the G matrix correlations for Pa. hamadryas (Hlusko and Mahaney 2009b; Hlusko et al. 2011), which demonstrate heritable, developmentally influenced components to tooth length and width measures. These similarities suggest that a P matrix based on tooth lengths and widths is a sufficient proxy for inferring the evolutionarily conserved developmental genetic processes and modules in the dental arch.

Linear measurements alone do not reveal how these genetic modules vary relative to one another within a species. To address this question, we used GM, a landmark-based approach to shape variation that is less constrained by traditional characterizations of the phenotype and captures the behavior of the entire tooth row.

GM analyses of shape variation along the tooth row reveal statistically indistinguishable primary and secondary dimensions of variation across these OWM taxa, in which concerted displacements of landmarks from the average dental arch shape correspond to tooth types, and therefore to the previously identified genetic modules for size traits (Hlusko and Mahaney 2009b), and to modules described by our P matrix correlations. The greatest dimension of shape variation across all OWMs reveals a pattern where the position of the incisors varies independently of the postcanine dentition, and the second greatest dimension shows an additional independence between premolar and molar positional variation. We interpret these concerted positional movements, or coordinated and regionalized changes in dental arch shape, to be evidence for phenotypic modularity corresponding to tooth type within the OWM dental arcade.

Our results indicate that at both the micro- (within-species) and macro- (across-species) evolutionary levels, genetic/developmental modularity influences variation in size and shape across the dental arcade of OWMs.


A significant caveat to studies of morphological integration is that measurements of adjacent structures are more likely to be correlated than measurements of more distant structures, confounding attempts to identify valid developmental modules (Mitteroecker and Bookstein 2007; Mitteroecker 2009; and also see Whiteley and Pearson 1899; Lewenz and Whiteley 1902, for Pearson's rule). Although this may present difficulty in structures such as the cranium or mandible, adjacency within the tooth row reflects actual developmental processes, given the serial homology of the dentition. We are confident that our results reflect the underlying biology because spatial autocorrelation cannot account for the entire pattern revealed in our analyses. For example, we found a degree of independence between the adjacent fourth premolar and first molar. We additionally found phenotypic independence between near-neighbor premolar and incisor teeth—a reflection of the lack of a genetic correlation between these teeth (Hlusko and Mahaney 2009b; Hlusko et al. 2011).

We argue that the inherent nature of a serially repeated structure provides significant power for identifying the underlying genetic and developmental mechanisms, as demonstrated by recent work on the vertebrate limb (e.g., Wagner and Gauthier, 1999; Young and Hallgrímsson 2005; Reno et al. 2008). Serial homologs also provide internal comparative controls for further testing hypotheses of selection and adaptation, evolvability, and the relative roles of genes and environment on developmental processes.


Our results of modularity by tooth type suggest that adult variation in the OWM dentition is influenced by early developmental processes. Although the genetic mechanism itself cannot be identified through phenotypic modularity, the existence of and kinds of interrelationships found through our approach pose specific hypotheses about development that readily complement gene-forward research. Phenotypic modularity studies can suggest developmental genetics experiments that more specifically target the mechanisms that pattern phenotypic variation and also make it available to selection.

Odontogenesis is perhaps one of the most well studied of the processes underlying skeletal phenotypes, with currently two models proposed for the genetic patterning mechanism of the dental arcade: (1) an odontogenic combinatorial code (Sharpe 1995; Thomas and Sharpe 1998), and (2) an inhibitory cascade model for relative molar sizes (Kavanagh et al. 2007) combined with a morphodynamic model for cusp patterning (Salazar-Ciudad and Jernvall 2002, 2010). The former relies exclusively on data from the mouse model and the latter is restricted to sources of variation within molars. Although the odontogenic code has been extrapolated to a more heterodont dentition (McCollum and Sharpe 2001), there is very little empirical evidence of how combinations of genes may or may not pattern a more heterodont mammalian dentition or different tooth types within that dentition.

Our combination of a quantitative genetic analysis (see Hlusko et al. 2006, 2011; Hlusko and Mahaney 2007b, 2009a; Koh et al. 2010) with a shape variation approach yields strong evidence that tooth types are genetic modules and provides some direction for further developmental study. As we noted, previous quantitative genetic results find evidence of incomplete pleiotropy between the molar and premolars, indicating that overlapping suites of genes influence their size variation. Our GM results bolster this interpretation in that within-species PC1 shows a correlated trajectory of shape variation between premolars and molars, whereas PC2 shows disparate patterns of variation between premolars and molars. One possible explanation for the submodule relationship of incomplete pleiotropy between premolars and molars could be that molar teeth are an extension of the primary dentition whereas premolars form as replacement teeth for the two deciduous premolars. As such, gene expression studies could be targeted at genes that may be expected to be differentially expressed in extensions of the primary dentition (the molars) in contrast to replacement teeth (the premolars).

A potential candidate for this could be a member of the Spry gene family, as mouse Spry2 and -4 knock-outs form teeth in the diastema (Klein et al., 2006), which have been interpreted as premolars (Kangas et al. 2004; Peterkova et al. 2005, 2006; Prochazka et al. 2010). However, Moustakas et al. (2011) found that the expression of Shh, Fgf, and Spry genes at cap stage is the same in mice and Monodelphis, and that these genes are also similarly expressed across the Monodelphis tooth classes (incisors, canine, deciduous premolars, premolars, and molars).

In contrast, Moustakas et al. (2011) found that Fgf10 is expressed in both the mesenchyme and epithelium of the Monodelphis domesticus canine, premolars, and molars but not in the epithelium of the incisors. This gene may thus prove useful for further investigation into the mechanisms underlying the independence between the anterior and postcanine dentition in OWMs.

A pattern of quantitative genetic independence between the anterior and posterior teeth has also been reported in mice (Hlusko et al. 2011). Therefore, the pattern of modularity identified in the OWMs may reflect a modular structure common to mammals more generally. Developmental genetics studies have shown a significant degree of mechanistic conservation in the developing dentition (Keränen et al. 1998, 1999; Jernvall et al. 2000; Järvinen et al. 2008; Nieminen 2009; Yamanaka and Uemura 2010; Moustakas et al. 2011). As teeth are also the best preserved element of the mammalian skeleton in the fossil record, any insights into the developmental mechanisms underlying mammalian dental variation—be they derived from phenotypic, genotypic, or developmental approaches—will likely improve our general understanding of mammalian origins and evolution over the last 160 million years (Luo et al. 2011). Recent studies combining QTL and “individual metrics of modularity” (Parsons et al. 2012) provide an emerging approach to further combine quantitative genetics with phenotypic integration studies to test these mechanistic hypotheses.


Given that the Cercopithecidae includes species with a wide range of sexual dimorphism (e.g., the male:female size ratios for these six species range from 1.02 to 1.95; body weight data from Rowe 1996), we find it remarkable that sex- and size-corrected tooth row data follow the same phenotypic trajectory as do the uncorrected data. This indicates that size, sexual dimorphism, and the shape variation that is theoretically independent of size and sex within the dentition all vary along the same first two principal components axes. We interpret this to be the result of the significant influence that a modular genetic framework has on dental development; variation is structured by this modular, developmental genetic organogenesis and as such varies along the specific morphological axes that result from it, much like the Lines of Least Evolutionary Resistance (LLER) as proposed by Schluter (1996).

The one notable exception to this pattern is the Asian colobine Pr. rubicunda (Fig. S3). The existence of a different sexual dimorphism trajectory for Pr. rubicunda from the other five cercopithecoid species in our study emphasizes that the results noted in the previous paragraph are not an artifact of our analytical approach—not all taxa follow the same “rule.” But this phenomenon begs the question of why this one species might have a different pattern.

The genus Presbytis (with 10–11 species and perhaps as many as 27 subspecies; Groves 2001; Meyer et al. 2011) is found across the Greater Sunda Islands, and is thought to have migrated there from mainland Asia in the Late Miocene–Pliocene and to have spread to the Mentawai Islands in the Plio-Pleistocene (Meijaard and Groves 2004). As is common for the plants and animals of this region, the multiple and contradicting reconstructions of the evolutionary history of Presbytis indicate that it was complex (as reviewed in Meijaard and Groves 2004; Harrison et al. 2006).

Of the two Presbytis species included here, Pr. rubicunda is found on the island of Borneo and Pr. melalophos on the island of Sumatra, with a molecular divergence date of approximately 1.3Ma (Meyer et al. 2011). These two species are similar in size and in having essentially no sexual dimorphism in body size, and there are no dramatic differences in adaptive regime (Rowe 1996 and references therein). As such, the difference we observe in how sexual dimorphism influences Pr. rubicunda dental variation is a conundrum. It has been suggested that the entire genus Presbytis has dwarfed since the Pleistocene (e.g., Harrison et al. 2006; N. Jablonski, pers. comm.). Given that different occurrences of dwarfism have resulted in different patterns of integration in the primate cranium (e.g., Frazier 2011), our findings suggest that dwarfism in different clades within Presbytis may be parallelisms and that these independent episodes differentially affected the ontogeny of sexual dimorphism in their dentitions.


Although the broad overarching pattern seen in our results helps to elucidate the pervasiveness of modularity within OWMs and perhaps mammals more generally, differences within our results provide insight into more specific questions about adaptation, selection, sexual dimorphism, and dwarfism.

Our GM results suggest that the canine may play an important role in influencing the relative positioning of the other tooth modules. Differences in canine shape vary with the curvature of the dental arch between species and subfamilies. Very little is currently known about the development of the canine, and, importantly, how systemic factors such as testosterone levels may affect the size and shape of this tooth. From our results, we propose that an investigation into the various factors that may affect the canine (e.g., body size dimorphism and hormone-sensitive responses) will have implications for how the modularity of the dentition has accommodated these influences. The fact that these canine shape differences are likely responsive to phenotypic changes outside of the dentition suggests that a canine module in OWMs (and likely all other primates) is biologically distinct from the incisor and postcanine modules.

We also find varying levels of integration across these six OWM species as measured by linear traits; although the pattern of high versus low correlations in the P matrices follows the pattern found in the G matrix of Pa. hamadryas (Hlusko and Mahaney 2009b), the average strength of correlations for these species is different. For example, we find lower overall phenotypic correlations within the Presbytis taxa, moderate correlations within Co. guereza and Ce. mitis, and high morphological integration in Pa. hamadryas and M. fascicularis. The biological significance of this, if any, remains to be explored, but may have implications for how readily various components of the dental phenotype can respond and have responded to selection.

In our standardized, common morphospace analyses conducted using SpaceAngle (Sheets 2002), we additionally detect significant phenotypic differences between Co. guereza (our least dentally dimorphic species) and several other taxa. This may indicate that other phenotypic dimensions of variation have become relatively more important in Co. guereza. As such, standardizing for sex and size may have revealed a distinct phylogenetic signal within colobine dental modularity, a promising direction for further interspecific comparisons.


Although primates have been the focus of a number of studies of phenotypic modularity (e.g., Lieberman 2004, and accompanying volume), our study is directly relevant to two that merit detailed comparison (Marroig and Cheverud 2001, 2005; Gomez-Robles and Polly 2012).

Gomez-Robles and Polly (2012) performed a GM analysis on individual postcanine teeth in hominins (following their taxonomy). From their phenotypic analysis, they conclude that there are significant levels of covariation between premolar and molar shape and that stronger integration exists among molars than among premolars. They interpret the existence of phenotypic covariation between premolars and molars to negate the hypothesis that there are separate premolar and molar fields in the postcanine dentition as previously proposed (Butler 1939; Dahlberg 1945; Harris 2003; Hlusko and Mahaney 2009b; Hlusko et al. 2011). However, from the perspective of our current phenotypic results and previous genetic work (Hlusko and Mahaney, 2009b; Hlusko et al. 2011), one would expect exactly the pattern Gomez-Robles and Polly (2012) found: more covariance within molars due to higher levels of pleiotropy than between premolars and molars due to their lower, but still detectable, pleiotropic effects.

Previous studies of morphological modularity corroborate the idea that modularity in complex phenotypes is a matter of degree (Klingenberg et al. 2003). The Gomez-Robles and Polly study was confined to a phenotypic analysis of the postcanine dentition, thus lacking information about how these teeth vary within the dentition as a whole. In light of this, we argue that their results are well aligned with ours and suggest that the genetic/developmental modularity we found characterizing OWM may well characterize the human lineage.

Our approach, which used phenotypic variation to test for modularity first demonstrated through quantitative genetics, follows the general approach of Marroig and Cheverud (2001, 2005). They estimated a G matrix characterizing skull shape variation for one genus of New World monkey (NWM; Cheverud 1996b) and then tested for similarities in the P matrices of 16 genera of NWMs. They found strong evidence for conservation of phenotypic covariance structure in skull traits across NWMs (2001). Similarly, we demonstrated that the covariance structure (in our case, a modular framework) is shared across the dental arcades of OWMs, representing 25 million years of evolution.

Marroig and Cheverud (2005) subsequently used the first PC of their NWM skull covariance data to demonstrate that extant diversification and morphological divergence occurred along a size trajectory that matches within-species variation—that micro- and macroevolution has generally occurred along the same trajectory. In this regard, our results differ from theirs. When the GM PC1 and PC2 data for these six OWM species are plotted in aggregate (Figs. 1, 2), the species all share the same within-species relationship between PC1 and PC2, but that the opposite relationship differentiates them (Fig. 6A). Therefore, although modularity is similarly influencing shape variation in all of the OWM species studied here, a different relationship between the modules distinguishes between them. This relationship between micro- and macroevolutionary modularity is discussed in more detail below.

Figure 6.

Macroevolutionary divergence (A) under a modular framework and (B) under LLER. Schematic of population variation and species phenotypic divergences in phenotypic space (PC1 and PC2 of the common morphospace). Ellipses represent individual species and their clouds of population variation, with the major axis defined by P-max (and G-max, as found for Marroig and Cheverud 2005, Schluter 1996, and this study.) Under our genetic modular framework, species diversify along a distinct axis while P-max vectors (within-species variation) parallel each other (panel A). Under LLER, species diversify along P-max (panel B).


From plants to animals, microevolutionary studies have used genotypic and phenotypic covariance matrices to identify modularity in morphology (e.g., Baumgartner, 1995; Albertson et al. 2003; Ashman and Majetic, 2006; Hulsey et al. 2006; Rosas-Guerrero et al. 2010). The traits used in these matrices are often chosen for their adaptive value (Roff 1996) and perceived diversity across a phylogeny. For example, studies in cichlid fish have shown decoupling of modules representing adaptive traits on a macroevolutionary scale (Albertson et al. 2003; Hulsey et al. 2006).

Our study is distinct from these in that we have demonstrated phenotypic modularity at micro- and macroevolutionary levels for traits that were not defined with respect to function, but rather were revealed independently through a GM approach and which are consistent with G matrix correlations and tooth genetic modularity. The modules we studied are stable across species with diverse social structures, dietary specializations, habitats, biogeography, etc., the factors by which most of these other studies (referenced above) seek to explain the modularity they observe.

Patterns of OWM dental variation have a conceptual relationship with the LLER as described by Schluter (1996). The LLER are proposed to constrain short-term evolution to the direction of greatest additive genetic covariance. In our study, this direction of greatest genetic covariance for size traits (i.e., G-max) looks like the direction of greatest phenotypic covariance (i.e., P-max) within species, and that this vector is the same across all species regardless of variation in habitat, diet, social structure, etc. However, our results do not conform entirely to Schluter's (1996) findings, as he assumes ancestry within the extant populations sampled and argues that the G-max additionally corresponds to the difference vector between species means and explains species divergences (Fig. 6B).

The OWM results are more aligned with Walker and Bell's position (2000) that incorporating information about the true ancestral morph can uncover different ancestor-descendent evolutionary trajectories than the line of maximal differentiation among derived, extant species. Indeed, as seen from aggregate PC plots (Figs. S1, S2, 6A), our extant OWM taxa do differentiate along a linear trajectory distinct from the trajectory that is common to them all. Determining what selective events have occurred during OWM divergence requires more detailed knowledge of the ancestral conditions of the clade, both in terms of habitat niche and ancestral morphology. Our next step is to incorporate fossil data into our research, to more directly parse the evolutionary influences of genetic modularity, divergent selection, and phylogenetic history on the OWM radiation.

Associate Editor: T. Streelman


The authors are grateful to all of the research assistants who have collected data for this project: J. Addiss, S. Akerson, L. Bates, J. Cohen, A. Holden, D. Lopez, T. Monson, K. Morrish, A. Murua-Gonzalez, K. Timmins, M. Watkins, J. Yoshihara, and A. Zowghi. We would also like to thank curators E. Lacey and C. Conroy (Museum of Vertebrate Zoology), E. Westwig (American Museum of Natural History), L. Gordon (National Museum of Natural History), and Y. Haile-Selassie and L. Jellema (Cleveland Museum of Natural History) for access to specimens. S. Amugongo, J. Carlson, M. Carr, D. DeGusta, N. Jablonski, M. Mahaney, C. Marshall, J. Shade, S. Sholts, E. Simms, T. White, and M. Zelditch also provided valuable discussion and feedback during the course of this project. We also want to thank two anonymous reviewers for their valuable comments and suggestions that greatly improved this manuscript. This work was supported by a grant from the National Science Foundation, BCS 0616308. It builds on previous research also funded by the National Science Foundation, BCS 0500179 and 0130277, and DDIG 9903435 (to A. Walker), and by the Human Evolution Research Center (UC Berkeley). OTR was partially supported by an National Science Foundation GRF, and TMG was partially supported by a Peabody Fellowship from the University of California Museum of Paleontology.