Among extant Anthropoidea, the platyrrhine Aotus stands apart as the only wholly or partially nocturnal species (Wright, 1996). Anatomical and genetic evidence from the visual system suggests that crown Anthropoidea were diurnal and that the nocturnal habits of Aotus are secondarily acquired (Martin, 1973, 1979, 1990; Cartmill, 1980; Jacobs et al., 1996; Ross, 1996, 2000). The antiquity of the diurnal to nocturnal transition in Aotus is not well understood. A key anatomical marker for nocturnality is an enlarged orbit. Nocturnal primates in general have relatively large orbits but nocturnal haplorhines (Tarsius and Aotus) have comparatively enormous ones, proportionately larger than nocturnal strepsirrhines of comparable size (Cartmill, 1980; Martin, 1990; Kay and Kirk, 2000; Kirk and Kay, 2004). This hypertrophy stems from the absence in Aotus and Tarsius of a tapetum lucidum, a reflective layer behind the retina (Kirk and Kay, 2004). In nocturnal strepsirrhines, the tapetum reflects photons back onto the retina, giving its photoreceptors a second opportunity to trigger the retinal photoreceptor array. The loss of a tapetum is hypothesized to have evolved as part of a package of adaptations for high visual acuity in stem haplorhines. In the absence of a tapetum, reacquisition of nocturnality in Tarsius and Aotus is thought to have selected for the evolution of a much larger eye and a larger array of photoreceptors than typically found in nocturnal forms. Therefore, the anatomical hallmark of nocturnality in an extinct anthropoid should be the presence of a very large orbit—larger than in nocturnal strepsirrhines.
Documenting the evolution of nocturnality in platyrrhines is hampered by the extreme rarity of fossil skulls. Only five crania of Cenozoic platyrrhines have been described: Tremacebus, Dolichocebus, Chilecebus, Homunculus, and Lagonimico. Of these, one (Lagonimico) is too crushed to allow a reliable estimate of orbit size and another (Chilecebus) is not yet fully described. The ∼ 20-Myr-old platyrrhine Tremacebus harringtoni (Fig. 1) is noted for its orbital enlargement (Rusconi, 1935; Hershkovitz, 1974), a feature that led Rosenberger (1979, 1984) and Szalay and Delson (1979) to conclude that it has phylogenetic affinities with Aotus and that nocturnality in the Aotus clade extends back at least to the early Miocene. However, though relative orbit size in Tremacebus falls within the range of nocturnal strepsirrhines, it is relatively smaller than the orbits of nocturnal haplorhines Aotus and Tarsius (Kay and Kirk, 2000). This led Kay and Kirk to conclude that the activity pattern of Tremacebus may not have been nocturnal. The only possible, if implausible, alternative interpretation is that Tremacebus was nocturnal and evolved a tapetum. Mohanamico hershkovitzi (Aotus dindensis) (Luchterhand et al., 1986) from 12.8-Myr-old levels in Colombia probably had large orbits also and therefore could have been nocturnal, although only a small part of the orbit is preserved (Setoguchi and Rosenberger, 1987).
What additional evidence can be brought to bear on the question of the antiquity of nocturnality in extinct platyrrhines? One possibility is to examine the olfactory system. In primates, the olfactory bulbs are relatively larger in nocturnal species than in their diurnal close relatives (Barton et al., 1995), and this trend is particularly obvious in Aotus. If Tremacebus had a particularly large olfactory bulb compared with other platyrrhines, this would provide some support for the hypothesis that it was nocturnal. Fortunately, the braincase of Tremacebus, when CT-imaged, reveals a partial natural endocast of the olfactory fossa, which housed the olfactory bulbs. In this article, we pose the question of whether the proportions of the endocast of the brain and of the olfactory fossa can prove useful in determining the relative proportions of the brain and olfactory bulb in an extinct anthropoid. If so, and if Tremacebus were found to have a relatively large fossa (and bulb), this would be evidence for nocturnality.
Before we examine the evidence pertaining to the size of the olfactory bulbs in Tremacebus, first we summarize some important aspects of the anatomy and genetics of olfaction and the relative olfactory abilities in haplorhines.
Olfactory sensory neurons (OSNs) are the primary sensory cells of the olfactory system. In mammals, there are 6–10 million of these cells. Cell bodies of OSNs' send single dendrites to the surface of the neuroepithelium (investing portions of the nasal septum and ethnoturbinals, of the ethmoid and the nasal septum) and terminate in a knoblike swelling. Arising from the swellings are olfactory ciliae that contain the sensory receptors for smell (Young, 1966; Firestein, 2001). Leading away from the OSN cell bodies are single axons that project to the main olfactory bulb (MOB) of the central nervous system. The MOB contains glomeruli. Over the entire array of the neuroepithelium, all of the neurons expressing the same odorant receptor gene converge on the same glomeruli in the MOB (Mombaerts, 1999; Rubin and Katz, 1999; Firestein, 2001). In the glomerulus, OSN axons meet dendrites of mitral and tufted cells that form the main projection cells from the olfactory bulb to higher-order olfactory structures (e.g., the piriform lobe) and other brain centers. The relay system from neuroepithelium to the mitral and tufted cells is modulated by intrabullar inputs and inputs from other parts of the brain (Shipley et al., 1995).
Mammalian odor receptors (ORs) comprise a single structural family of G-protein-coupled receptors (Firestein, 2001; Godfrey et al., 2004; Malnic et al., 2004). In mice, ∼ 1,000 genes code for ORs, making this the largest family of genes in the mammalian genome (Godfrey et al., 2004). Each OR codes for the recognition of a particular feature of a ligand (odorant molecule; mostly low-molecular-mass organic compounds) (Firestein, 2001). An odorant molecule may have more than one recognizable part (a determinant), and a particular OR can detect a common chemical structure on a variety of odorant molecules. Further, each OR can have its response to a particular determinant reduced by an antagonist responding to a related structural compound. Peripheral coding is based on activation of arrays of olfactory receptor cells with overlapping tuning profiles (Duchamp-Viret et al., 1999). If odorant molecules are recognized by more than one receptor, and if this response can be modulated by the presence of a structurally related molecule, it is easy to see why the olfactory system is capable of such fine sensitivity and discrimination (Firestein, 2001).
Many psychophysical tests have been designed to test odor sensitivity in mammals where sensitivity refers to the threshold of stimulus at which response occurs more often than chance. A challenge for such sensitivity tests is that individual ORs cannot really measure concentration. As concentration of an odor is increased, additional glomeruli are recruited (Rubin and Katz, 1999). A further implication of the olfactory system's structural design is that different individuals or species may be better at discriminating or detecting odor not only due to differences in the number of ORs, but also to what kinds of ORs and, downstream, how many different kinds of glomeruli they possess.
A second olfactory detection system, the vomeronasal system (VNS), is separate from the main olfactory system. It responds to pheromones generally (but not always) produced and emitted by conspecifics. Pheromones act as important chemical signals in mating behavior and other social interactions. One critical difference is that in the VNS, the sensory lumen of the vomeronasal organ (VNO) is filled with fluids of the vomeronasal gland and receptive to the dispersal of odorant molecules through fluid, whereas the main olfactory epithelium is receptive to odorant molecule dispersal through air; also, the VNO epithelium is nonciliated (Keverne, 1999). The VNO is surrounded by an erectile tissue that functions as a physiological pump (Salazar et al., 1997) to suck into its fluid-filled lumen high-molecular-weight odorant molecules such as pheromones (Del Punta et al., 2002). The VNO olfactory neuron axons project both to the accessory olfactory bulb and toward the hippocampus, but not to the main olfactory bulb. Its glomeruli are located caudal to the main olfactory bulb. Projections from the accessory olfactory bulb in turn mediate responses in the endocrine system. Two families (and more than 200 receptors) of G-protein-coupled receptors, unrelated to OR families, have been identified in the VNO of mice (Firestein, 2001). As noted above, all ORs expressing a particular receptor over the entire array of nasal neuroepithelium converge on a single target, one or at most three glomeruli, in the olfactory bulb. In contrast, VNO receptors disburse to as many as 10–30 glomeruli in the accessory olfactory bulb. Sexual dimorphism of the VNS has been established in several vertebrate species, and often certain structures of this system will be larger in males, while others will be larger in females (Halpren and Martinez-Marcos, 2003). However, both sexes appear to have almost the same expression of VNO receptor genes, suggesting that they may detect similar pheromones and that differences in behavioral response between sexes are the result of higher brain functions in the VNS (Firestein, 2001).
What evidence is there for variations in the odor-detecting abilities in the olfactory system and vomeronasal system of primates? Several lines of evidence contribute a partial answer to this question but not all lines of evidence are concordant.
Two (linked or nonindependent) aspects of the nervous system are thought to relate in some way to the odor-detecting abilities of primates: the degree of development (surface area and number of odorant receptors) of the sensory neuroepithelium within the nasal cavity where olfactory receptors reside, and the size of the olfactory bulb and other brain components that relay and integrate olfactory information (e.g., the piriform lobe). By analogy, two anatomical components of the vomeronasal system are the size of the vomeronasal organ and the size of the accessory olfactory bulb. An ancillary measure of the importance of either system would be the presence of scent glands or other scent-related structures that animals use to communicate with conspecifics or other species.
Olfactory epithelium and MOB.
The olfactory epithelium is located on the ethmoid turbinals and nasal septum. Turbinals of strepsirrhines generally are more elaborate than those of haplorhines (Paulli, 1900; Cave, 1973), but we know of no mensurational data comparing the development of the turbinals of strepsirrhines or haplorhines. Also, the surface area of the turbinals includes both the respiratory and the olfactory epithelium. The main function of the turbinal bones is believed to be that of warming and moistening incoming air (Young, 1966; Van Valkenburgh et al., 2004), and the olfactory epithelium only covers the more posterior turbinals, formed by the ethmoid bone, toward the back of the nasal cavity. The olfactory epithelium also extends onto the nasal septum (Young, 1966; Ewer, 1973; Koppe et al., 1999; Menco and Morrison, 2003). This means that the surface area of the turbinals as a whole is unlikely to provide any precise information about olfactory sensitivity, although the ethmoturbinals may do so.
The volumes of the main and accessory olfactory bulbs and other brain components related to the olfactory system have been reported in a broad spectrum of mammals, including many primates (Stephan et al., 1981, 1984; Baron et al., 1983; Frahm et al., 1984). The relative size of the main olfactory bulbs relative to brain size or body weight (Fig. 2) is smaller in haplorhines than in strepsirrhines but does not differ in a systematic way between platyrrhine and catarrhine primates. Indeed, in the region of brain size overlap, MOB size in platyrrhines and catarrhines overlaps extensively. Among diurnal anthropoids, the only significant outlier is Homo sapiens (not shown in Fig. 2), a species that has exceptionally small olfactory bulbs (Baron et al., 1983).
The volume of the olfactory bulb in platyrrhines is highly correlated with overall brain volume and body weight (Fig. 3). Scaling coefficients, slopes of least-squares regressions on logged data, are less than 1.0, indicating a negative allometry between olfactory bulb size and either brain volume or body weight. Among haplorhines, the nocturnal species Tarsius and Aotus have proportionally larger olfactory bulbs than diurnal anthropoids. This is obvious comparing a lateral view of the brains of two similar-sized platyrrhines, Aotus and Callicebus (Fig. 4).
In an allometrically and phylogenetically controlled comparison of the visual and olfactory components of the brain, Barton et al. (1995) conclude that across all primates, there has been an evolutionary trade-off between specializations in the olfactory and visual systems. Thus, they report that the sizes of structures of the brain that process olfactory information are negatively correlated with the sizes of brain structures related to vision. For example, species with large olfactory bulbs have relatively smaller striate visual cortices.
The relative size of olfactory and visual components varies with respect to activity pattern (Barton et al., 1995). Nocturnal clades tend to have larger olfactory structures and smaller visual structures than do diurnal or cathemeral clades. Within primates, using the method of independent contrasts (Felsenstein, 1985; Purvis and Rambaut, 1995), Barton et al. (1995) identify four evolutionary shifts (transitions) between nocturnality and diurnality: anthropoids became diurnal from a nocturnal primate ancestor; the nocturnal platyrrhine Aotus had a diurnal ancestor; the diurnal clade of Propithecus + Indri evolved from a nocturnal ancestor; and the diurnal/cathemeral clade of Eulemur + Varecia evolved from a nocturnal ancestor. They report that olfactory bulbs and other olfactory components become larger in the nocturnal transitions and smaller in the diurnal transitions. The reverse pattern occurs with respect to the visual system. Visual brain components become smaller in nocturnal transitions and larger in diurnal transitions.
Folivorous and frugivorous anthropoids also show differences in the visual system when the effects of phylogeny are controlled for. Folivorous diurnal anthropoids have smaller visual cortices than frugivorous diurnal species. However, independent contrasts in dietary preference are not significantly correlated with contrasts in the size of the olfactory system (Barton et al., 1995).
Vomeronasal organ and accessory olfactory bulb.
In mice and many primates, chemoreceptors located in the vomeronasal organ are exposed to nonvolatile odorant molecules that diffuse through a fluid medium, via the engagement of a physiologically regulated pump mechanism (Firestein, 2001). Axons of the vomeronasal neurons project to the accessory olfactory bulb located at the dorsolateral limit of the main olfactory bulb.
The vomeronasal organ and anterior accessory bulb are well developed in strepsirrhines (Stephan et al., 1981, 1984; Baron et al., 1983). The organ and bulb are smaller and extremely variable in platyrrhines (Stephan et al., 1981, 1984; Baron et al., 1983; Dennis et al., 2004) and rudimentary in adult catarrhines (Stephan et al., 1981, 1984; Baron et al., 1983). This suggests that pheromonal communication may not be as important to platyrrhines compared to strepsirrhines and may be even less important in catarrhines than in platyrrhines. However, it is important to note that the dissociation of the vomeronasal system and main olfactory system is not as absolute as generally thought. Recent research suggests that in certain species, the main olfactory system (MOS) can detect pheromones, and the VNS can detect volatile chemicals (Doving and Troiter, 1998; Dulac, 2000; Rodriguez et al., 2000; Smith et al., 2001; Zhang and Webb, 2003). This may mean that catarrhines, while lacking a VNO, may still rely to some extent on pheromonal communication mediated by the main olfactory epithelium.
A final word of caution. After an extensive literature review, we find ambiguous data to support the hypothesis that species with well-developed olfactory bulbs (either relatively or absolutely) are necessarily endowed with better olfactory sensitivity and/or discrimination. While common sense suggests that the relative or absolute size of the sensory nasoepithelium or olfactory bulbs must be related to differences in olfactory abilities, the literature offers contradictory evidence. In addition, glomeruli in the olfactory bulbs may vary in size within and across species (K.B. Doving, personal communication); this indicates that it may be inaccurate to rely on bulb size comparisons across species to indicate functional capabilities and the range of odorants detected in those species. As discussed below, only detailed behavioral tests may begin to answer these questions. Also, as will be apparent in the next section, newly emerging data on the genetics of olfactory receptors provide some challenging discrepancies from expectations based on the anatomy of the brain.
Genes that code for olfactory receptors in mice and other mammals can be subdivided into subfamilies. OR genes are defined as belonging to the same subfamily if they code for amino acid sequences with ≥ 60% identity. This cutoff has functional significance because olfactory receptors that are 60% or more identical tend to recognize odorants with related chemical structures (Firestein, 2001).
Mouse and human OR genes have been studied extensively (Young and Trask, 2002; Godfrey et al., 2004; Malnic et al., 2004). Most recently, it has been reported that mice have 913 intact OR genes divisible among 241 subfamilies (Godfrey et al., 2004), whereas humans have 339 intact OR genes divided among 172 subfamilies (Malnic et al., 2004). Thus, mice have approximately 2.7 times as many ORs as humans but only 1.4 times as many subfamilies. Mice and humans share a substantial number of subfamilies but in most cases (89%) the number of ORs in a shared subfamily is greater in mouse than in human (Godfrey et al., 2004). It is possible that the number of subfamilies may be a good predictor of the diversity of odorant features that a species can detect (Godfrey et al., 2004), and the number of genes in each subfamily may well indicate the acuteness of that species' ability to detect odorants coded by that subfamily. It appears that intact humans possess genes from most subfamilies, which means that humans may be able to detect as wide a range of odorants as mice, but that mice may be more sensitive to odors and perhaps are better able to distinguish closely related odors. However, one complication in using the number of intact OR genes to predict olfactory sensitivity is that intact OR genes may not imply gene expression and, therefore functionality.
Finally, Godfrey et al. (2004) note that while mice and humans share 150 OR subfamilies, 13% of human OR subfamilies are not found in mice and 35% of mouse OR subfamilies are not found in humans (Fig. 5A). This raises the possibility that these species-specific OR subfamilies can detect odorants that are sensed by humans or by mice, but not by both.
One indirect way to compare olfactory abilities between species might be to compile the complete set of intact genes for each species. However, because the full set of OR genes is presently known only for a small number of species (e.g., humans, mice, rats, and dogs), calculating the number of intact OR genes must rely at the moment on a sampling approach. Rouquier et al. (2000) hypothesize that reduction in the sense of smell observed in primates could parallel the reduction in the percentage (not the absolute number) of intact OR genes. Therefore, comparative approaches to the study of ORs in primates so far have relied on the demonstration that a variable percentage of ORs carry one or more disruptions in the coding regions and are therefore noncoding, or pseudogenes (Rouquier et al., 2000). Rouquier et al. (2000) report that the percentage of pseudogenes in the OR gene repertoire varies considerably in primates. In their sample, the percentage of intact OR genes in the platyrrhines Callithrix and Saimiri ranges from 93% to 100%. In cercopithecids (Papio and Macaca), it is 65–81%; in nonhuman hominoids (Hylobates, Pongo, Gorilla, and Pan), it ranges from 50% to 61%; and in humans, it is just 30%. They hypothesize that the sense of smell varies as a function of the fraction of intact OR genes in the genome.
Additional attempts have been made to examine intact versus pseudogene percentages. Gilad et al. (2003) sequenced a random sample of 50 OR genes in different primates and determined the percentage of these genes that are intact. The percentage ranged from 65% to 72% in great apes and Macaca and 48% in Homo. Gilad et al. (2004) estimate the proportion of OR pseudogenes in 19 primate species. Using degenerate primers and PCR amplification, they assembled a random sample of 100 OR genes in each species and calculated the percentage of the sample that is composed of pseudogenes. They report 49% intact OR genes in humans. The greater and lesser apes range from 66% to 68%; cercopithecids range from 69% to 74%; and all but one platyrrhine fall between 83% and 85%. The exception among platyrrhines is Alouatta, which has 69% intact OR genes, comparable to catarrhines. The authors suggest that Alouatta resembles catarrhines because of its greater reliance on color vision compared with other platyrrhines. Alouatta and catarrhines have trichromatic color vision, whereas in other platyrrhines males are dichromatic and females are variably trichromatic or dichromatic.
Several things need to be resolved before the functional meaning of this genetic data can be understood. To begin with, as noted above, it is unclear whether intact OR genes imply functionality. A second problem relates to the reliability of using OR gene sampling to assess the number of intact OR genes in a species. The percentage of pseudogenes is hypothesized by Gilad et al. (2004) to give an unbiased estimate of the size of the intact OR gene family. An underlying assumption is that all species have roughly the same number of OR genes. However, they may not, which means that the percentages may not accurately represent the number of distinct kinds of OR genes within and across species. To illustrate this, suppose that mice have 82.6% intact genes while humans have 49% intact genes [supplemental data from Gilad et al. (2004)]. If each had the same number of OR genes, then mice would have 1.69 times as many intact OR genes as humans. But there are 636 OR genes in humans and 1,209 OR genes in mice (Fig. 5B) (Godfrey et al., 2004; Malnic et al., 2004). Using the percentages given by Gilad et al. (2003) gives us 999 intact OR genes in mice but only 312 intact OR genes in humans, with mice having 3.2 times as many intact OR genes as humans.
In short, because we do not know exactly how many OR genes, either intact or pseudogenes, exist in any nonhuman primate, the percentages given by Gilad et al. (2004) may underestimate the actual number of intact genes by an unknown and possibly large factor. The distribution of these intact genes into gene families is also unknown and, as discussed above, olfactory ability and sensitivity may rely not only on the number of intact OR genes, but also on how these intact genes are distributed into OR gene subfamilies. Thus, olfactory ability may not simply be a matter of the number of intact genes a species possesses, but also how these intact genes are distributed across subfamilies (Fig. 5).
Another troubling feature of the genetic findings is the apparent lack of correlation with the relative size of the olfactory apparatus. The data from Rouquier et al. (2000) indicate that humans have the lowest percentage of intact OR genes. The other taxa examined are arranged relative to humans as follows: humans < other hominoids = Eulemur < papionins < platyrrhines = mice. But the relative size of the olfactory bulb (from small to large) is human < hominoids = papionins = platyrrhines < strepsirrhines < mice. It is surprising that the strepsirrhine Eulemur should have comparable percentages to Macaca and that both should exceed platyrrhines, but no more surprising than that the platyrrhine Callithrix and the mouse are similar despite the vast difference in the relative sizes of MOBs in these species. This means that if the sampling paradigm of Rouquier et al. (2000) is valid [a point challenged by Gilad et al. (2004)] and if intact OR genes translate into olfactory ability mediated by the MOB, some strepsirrhines have a less well developed MOS sense of smell than either platyrrhines or mice, both of which would seem to be similarly endowed.
The genetic data from Gilad et al. (2004) is also discrepant with anatomical observations. In their data, humans have the lowest percentage of intact genes (relative to humans): humans < other hominoids ≤ cercopithecids < platyrrhines (except Alouatta) = Eulemur = mouse. But platyrrhines and catarrhines have similar-sized olfactory bulbs and both have much smaller bulbs than strepsirrhines. Likewise, the proportion of OR psuedogenes in the mouse and in the strepsirrhine Eulemur falls within the range of platyrrhines despite having far larger olfactory bulbs. Finally, among platyrrhines, Gilad et al. (2004) identify Alouatta as an outlier but Aotus as being similar to other platyrrhines, whereas the anatomical data have Alouatta as being unremarkable in terms of olfactory system size while nocturnal Aotus as having large bulbs. The two sets of genetic data conflict with one another and with the relative size of MOB. It should be noted that in the species mentioned here, the anatomical evidence appears to correlate more precisely with admittedly sparse behavioral data on olfactory ability and reliance. Perhaps the discrepancy is related to the way the comparisons are expressed, i.e., as percentages rather than absolute numbers. Until we know how many OR genes (pseudogenes plus intact genes) each taxon has, the percentages of pseudogenes may be misleading.
Lastly, as touched on above, we know very little about the expression of OR genes. Clearly, a single OR gene must code for many copies of a particular type of OR, but there may be differences in the number of copies in various species with attendant differences in olfactory acuity. With 1,000 OR genes, Firestein (2001) notes, the number of possible combinations of scents that could be recognized amounts to billions. The genetic evidence brought forward so far appears to have no bearing on the ways OR signals are mediated or recombined in the olfactory bulb or in higher brain centers. Clearly, we have a long way to go before either the anatomical or the genetic data can be used with confidence for comparisons of the olfactory abilities among primate species.
We end this section with a final short note concerning genes coding for receptors of the vomeronasal system. Recent comparative studies indicate that the genes for expression of VNO receptors are present in strepsirrhines and platyrrhines but have mostly become pseudogenes in catarrhines. In the human genome with the exception of just a few genes, all putative genes that code for vomeronasal receptors are pseudogenes (Meredith, 2001; Liman and Innan, 2003). The loss of intact VNS genes is suggested to have occurred in synchrony with a greater reliance on vision and visual cues in communicating with conspecifics (Liman and Innan, 2003; Zhang and Webb, 2003). As noted by Gilad et al. (2004), Alouatta's higher percentage of OR pseudogenes is hypothesized to correspond to the acquisition of full trichromatic vision to aid in foraging; however, Alouatta resembles other platyrrhines in having a functional VNO system (Zhang and Webb, 2003; Webb et al., 2004). Webb et al. (2004) point out that this evidence indicates that the acquisition of color vision may not necessarily be the only variable responsible for the reduction of the VNS in primates. They suggest that additional ecological variables may have resulted in the reduction of this system. Certain environments may make leaving pheromonal scent marks difficult. However, if pheromone detection has been taken over by MOS in catarrhines, then perhaps like Alouatta, these species use color vision primarily for foraging tasks rather than conspecific communication. It is also possible that color vision may be used and perceived in different ways across species and in combination with signals from other modalities.
While there has been much written on anatomical differences of olfactory bulb (OB) size and on interspecific olfactory genetic repertoires, it is only possible to understand the importance of these findings in a species-specific functional way by examining olfactory behaviors in the species of interest. If the importance of olfactory ability to behavior in a given species can be correlated with a high number of intact olfactory genes or with large OBs (or the lack of olfactory ability and behavior with small OBs and many pseudogenes), then researchers may begin to use bulb size or genetic repertoires to predict olfactory ability in a given extant, or even extinct, species.
Before going further, it is important to keep in mind that olfactory ability or the sense of smell in a given species is most probably controlled by both the MOS, which senses odorants, and the VNS, which senses pheromones. Individual species will undoubtedly vary in the extent to which each system is used in their species-specific sense of smell.
In platyrrhines, olfactory ability plays an important role in behavioral responses and social interactions (Epple, 1986). This is especially true in behavior associated with reproduction, such as in the assessment of reproductive status (Hennessy et al., 1978; Boinski, 1992; Ziegler et al., 1993; Converse et al., 1995; Heymann, 1998; Smith and Abbott, 1998). Chemical stimuli have also been implemented in the suppression of subordinate female's ovarian cycles (Abbott, 1984; Epple and Katz, 1984; French et al., 1984; Heistermann et al., 1989; Barrett et al., 1990). Scent delivered via scent marks or urine may also be used in the recognition of conspecifics (Laska and Hudson, 1995), to communicate dominance, mediate interactions among individuals of different ranks (Epple et al., 1986), and may be important in interactions between infants and other group members (Kaplan and Russell, 1974; Epple et al., 1986), as well as in infant-mother bonding (Kaplan et al., 1979). Scent also appears to be important in territorial defense, in intergroup relations, and for maintaining intergroup spacing (Epple, 1974; Ueno, 1994b, 1994c; Smith et al., 1997; Lazaro-Perea et al., 1999), as well as for the discrimination of food items (Ueno, 1994a).
Several lines of evidence suggest that platyrrhines depend on olfaction more than catarrhines. For example, there is ample anatomical evidence in the form of scent glands as well as behavioral evidence of animals responding to urine and scent marks left by conspecifics (Epple, 1974; Uenor 1994a, 1994b, 1994c; Smith et al., 1997; Lazaro-Perea et al., 1999). Though it has yet to be tested systematically, these behaviors are likely to depend on the VNS and are probably mediated via pheromones. These observations are consistent with the presence of VNO and AOB in platyrrhines, whereas these structures appear to be rudimentary or absent in catarrhines.
While these examples establish the importance of the VNS in the behavioral repertoires of platyrrhines, there is also evidence that platyrrhines rely on odorants encoded by the main olfactory system. For example, the recognition of food odorants is likely to be a good indicator of the reliance on, and the sensitivity of, a species' main olfactory system. By examining behaviors related to foraging in the wild and by undertaking laboratory studies examining sensitivity to food-related odorants, researchers may begin to determine to what extent platyrrhines and other primates rely on the MOS. Recent studies show that anthropoids discriminate and remember different odorants, especially in the detection of food (Laska and Hudson, 1993a, 1993b; Ueno, 1994a, 1994b, 1994c; Garber and Paciulli, 1997; Laska et al., 2003b). Laska and Hudson (1993b) even found that in the squirrel monkey, different components in odor mixtures interact and can have complex relationships so that small changes in the composition of an odorant may have significant consequences on how that odor is perceived and interpreted as a signal.
A handful of comparative behavioral studies using olfactory cues mediated provide additional information on functionality. Evidence indicates that dietary preference rather than phylogeny determines how well both platyrrhines and catarrhines perform on olfactory tests; and these results appear to correlate with some of the genetic and anatomical observations. Laska et al. (1993b) found that both the platyrrhine Ateles geoffroyi and the catarrhine Macaca nemestrina can discriminate between objects on the basis of odor cues, can transfer their selections to new positive and negative stimuli, and can remember the significance of previously learned odor stimuli over prolonged intervals. However, with regard to the speed of initial task acquisition and the ability to master transfer tasks, A. geoffroyi outperformed M. nemestrina. This difference in behavioral ability in the platyrrhine Ateles correlates only weakly with OB size. The two have similar-sized bulbs, although in Ateles the AOB comprises 2.5% of the total size of the olfactory bulb, whereas in Macaca the percentage is zero. On the other hand, Laska et al. (2003a) found that the catarrhine M. nemestrina generally outperformed the platyrrhine Saimiri sciureus in detecting aliphatic aldehydes, a class of odorants that is presumed to indicate a fruit's degree of ripeness. This is in agreement with the findings of Laska (2001) that these two species exhibited different food preferences; compared with Siamiri, Macaca prefers foods with a higher carbohydrate and fructose content. This behavioral evidence correlates more clearly with anatomical evidence available for these two species, with Macaca having relatively much larger bulbs than Saimiri.
In addition to dietary preference, MOS ability has also been linked to activity pattern. Bolen and Green (1997) found the nocturnal Aotus monkey to be more adept at locating baited sites than the diurnal Cebus. Bicca-Marques and Garber (2004) also found that in foraging tasks Aotus relies more on olfactory cues than Callicebus, or most Saguinus. These differences are hypothesized to relate to the reduced availability of visual cues in nocturnal species, which leads to a greater reliance on olfactory ability. The postulated differences in olfactory abilities goes along with the larger size of the olfactory bulbs in Aotus but has no corresponding difference in the percentage of pseudogenes.
By gathering behavioral evidence in careful studies, such as in the ones mentioned here, researchers may begin to understand exactly what olfactory genes and olfactory bulb size can tell us about how a species may have lived, e.g., whether it was nocturnal or diurnal, and even whether it ate fruit or leaves (although the latter has not yet been demonstrated) (Barton et al., 1995).
MATERIALS AND METHODS
The type specimen of Tremacebus harringtoni consists of a partial skull (Figs. 1 and 6). Carlos Rusconi received the specimen in 1932 from Thomás Harrington, who collected it together with other fossil mammal remains of Colhuehuapian age (∼ 20 Ma) from approximately 12 km southwest of Cerro Sacanana, in north central Chubut Province, Argentina. Rusconi named the specimen a species of Homunculus (Rusconi, 1933) and provided an extended description of the fossil (Rusconi, 1935). Hershkovitz (1974) proposed a new genus for this skull. At some point after Rusconi's original description, a substantial amount of plaster was added to reconstruct missing parts. Later, Hershkovitz and others made an effort to have the plaster removed but succeeded only partially because the plaster is cleverly tinted to match the color of the fossil bone. In an effort to determine more precisely the limits of bone, matrix, and plaster, and to better appreciate the structural details of the interior of the skull, one of the authors (R.F.K.) borrowed the specimen from Tucuman and had it CT-imaged. Details of the preservation provided by Hershkovitz (1974) are substantially correct except that the apex of the orbit and optic foramen are not preserved as Hershkovitz claimed, and the lateral pterygoid plate figured by Hershkovitz consists of plaster. Hershkovitz's claim that there was a large inferior orbital fissure in life cannot be confirmed.
A sample of CT images of extant platyrrhines was the source of comparative measurements from the interior of the cranium (Table 1). Data for brain size and the size of the main olfactory bulb and accessory olfactory bulb come from previously published information (Stephan et al., 1981, 1984; Baron et al., 1983; Frahm et al., 1984).
Measurements are in millimeters. Olfactory fossa breadth and maximum endocranial breadth are illustrated in Figure 7. The estimate of relative size of the olfactory fossa is based on measurements of olfactory fossa breadth and maximum endocranial breadth. A least-squares regression was fit to In maximum endocranial breadth (MEB; independent variable) versus In olfactory fossa breadth (OFB; dependent variable) for eight species of diurnal platyrrhines. The equation expressing this line is In OFB = 0.794 (In MEB) − 1.557. For each taxon, the expected OFB was calculated from this equation. The observed (measured) OFB for each species was compared with the expected and expressed as a residual (OFI): OFI = 100 × (observed − expected)/(expected). The Rusconi Collection at Museo de Fundación Miguel Lillo, Tucuman, Argentina. The collection label and catalog entry for the skull of Tremacebus in the Rusconi collection and the number painted on the specimen gives the number as 619, but Rusconi (1933, 1935) refers to it as number 661.
The Tremacebus specimen was scanned at the High-Resolution X-Ray Computed Tomography Facility at the University of Texas at Austin, which is described by Ketcham and Carlson (2001). X-ray energies were set to 150 kV and 0.16 mA using a FeinFocus X-ray source. X-ray intensities were measured using an Image Intensifier detector employing a 1,024 × 1,024 video camera. Each slice was acquired using 1,000 views (angular orientations), with four samples taken per view. The specimen, mounted in a plastic cylinder, was scanned with a centered axis of rotation (Ketcham and Carlson, 2001) with a source-to-object distance of 135 mm (Ketcham and Carlson, 2001). Slice thickness and interslice spacing was 0.0466 mm (one video line). The image field was reconstructed to 43 mm, based on a maximum field of view of 44.164 mm, yielding an interpixel spacing of 0.042 mm. Reconstruction parameters were calibrated to maximize usage of the 16 bit range of grayscales available in the output images. Twenty-seven slices were acquired for each rotation of the turntable, with a resulting acquisition time of about 11.1 sec per slice. The entire scan took about 4.75 hr, including calibrating the scanner and mounting the specimen, which took about 1 hr. The data comprises 1,177 slices, from the front to the back of the skull.
The coronal slice-by-slice animation may be viewed on the DigiMorph Web site (http://www.digimorph.org/index.phtml). The coronal movie (COR) begins at the tip of the left premaxilla; the slices are in anterior view. The horizontal movie (HOR) starts dorsally and passes ventrally; the slices are oriented in dorsal view. The sagittal movie (SAG) proceeds from left to right through the specimen; the slices are in left lateral view.
The platyrrhine skulls that comprise the comparative database were scanned with various protocols, depending on skull size, in order to maximize resolution for each specimen. Resolution ranged from 0.088 to 0.176 mm in the coronal plane (interpixel spacing) and 0.097 to 0.189 mm in the Z-axis (interslice spacing). For further details, see Rossie (2003).
Perhaps the most comparable osteological measurement of the size of the olfactory bulbs would be to determine the volume of the olfactory fossa. However, in Tremacebus only the dorsal portion of the natural endocast of the olfactory fossa is preserved and some portions of the braincase are missing and cannot be mirror-imaged. Therefore, we used the breadth of the fossa as our measure of olfactory fossa size and estimated maximum breadth of the endocranium (in the parietal region) as a measure of brain size. The validity of this approach is examined below. Serial CT cross-sections in the coronal plane were examined and two images selected for each specimen: one at the point where the internal surface of the neurocranium corresponding to the maximum breadth of the fossa containing the olfactory bulbs, and a second at the level where the brain would have attained its maximum breadth, usually in the parietal region. The measurements are illustrated in the platyrrhine Pithecia sp. in Figure 7. Each section was imported into Photoshop, version 7.0 for Macintosh, and the maximum breadth of the fossa (OFB) and parietal endocranium (PB) were measured in pixels using the measurement tool, then corrected for the number of pixels per mm. The breadth across the neurocranium in Tremacebus (Fig. 8) was estimated by mirroring missing parts of the right side onto the right.
Our objective is to draw inferences about the relative size of the olfactory bulb and brain size of Tremacebus, but all we have are osteological measurements of the interior of the cranium. The first step is to see whether olfactory fossa breadth and maximum endocranial breadth can be used as proxies for olfactory bulb volume and brain volume.
Species means of endocranial dimensions for extant and fossil taxa are presented in Table 1. For the two osteological measurements to serve as a reliable surrogates for the brain volumes, it must be shown first that each osteological measurement is highly correlated with its soft-tissue surrogate (i.e., olfactory bulb volume with olfactory fossa breadth, and brain volume with maximum endocranial breadth) and second, that the deviations away from the general trend are matched. For our sample of eight diurnal anthropoid species, ln (natural log) olfactory bulb volume and ln brain volume are tightly correlated: 67% of the variance in ln olfactory bulb volume is explained by ln olfactory fossa breadth in a group that includes platyrrhines and Miopithecus (the only catarrhine in our sample; P < 0.007). Notably, the olfactory bulb volume of Aotus is larger than other anthropoids (Fig. 9A). The same trends are apparent in the osteological measurements. Among diurnal anthropoids, ln olfactory fossa breadth and ln maximum endocranial breadth are also correlated: 75% of the variance in brain volume is explained by maximum endocranial breadth (P < 0.002). Again, olfactory fossa breadth of Aotus is larger than other platyrrhines (Fig. 9B).
Residuals (a percentage of expected) are calculated from the least-squares regression of ln brain volume (independent variable) versus ln olfactory bulb volume (dependent variable). Similar residuals (called olfactory fossa index; OFI) are calculated between ln maximum endocranial breadth (independent variable) and ln olfactory fossa breadth (dependent variable). The residuals of the two comparisons are significantly correlated: P < 0.03 [Fig. 10; the Spearman's rank correlation (rho) between the residuals is 0.77 with a P value of 0.016; the Kendall rank correlation (tau) is 0.556 with a P value of 0.037]. From this we conclude that the osteological measurements provide reasonable surrogates for the brain measurements: species with relatively large olfactory bulb volumes also have relatively large olfactory fossa breadths. Thus, an inference about relative olfactory bulb size from endocranial dimensions is warranted in fossil platyrrhine skulls.
From the cranial dimensions of Tremacebus, it is probable that the olfactory bulb of this species does not depart from that of most extant platyrrhines of similar size. The OFI is similar to that for Callimico (Fig. 10). This gives an estimated olfactory bulb size residual of ∼ +12, most resembling Pithecia in our sample and unlike Aotus which has comparatively large olfactory bulb (a residual of +19). Admittedly, the samples are small, but on the available evidence it seems reasonable to infer that Tremacebus did not have as well developed olfactory bulbs as Aotus. More specifically, given that Aotus is nocturnal whereas other platyrrhines are diurnal, the smaller inferred relative size of its olfactory bulbs suggests that Tremacebus was diurnal.
The anatomical and genetic distinctiveness of the vomeronasal system from the main olfactory system is well known. Though there is evidence that in some species the MOS may detect pheromones and the VNS may detect odorants (Doving and Troiter, 1998; Dulac, 2000; Rodriguez et al., 2000; Smith et al., 2001; Zhang and Webb, 2003), in general the two systems appear functionally distinct, with the former playing a role in the detection of nonvolatile pheromones and the latter detecting volatile compounds. While foraging, animals appear to rely on volatile cues and therefore on the main olfactory system. Thus far, the anatomical evidence of OB size in platyrrhines and catarrhines appears to correlate with olfactory behavioral differences mediated by the MOS (e.g., foraging activity).
As touched on in this article, with respect to foraging, there is conflicting evidence as to whether catarrhines or platyrrhines rely more on the olfactory sensory modality (Laska et al., 2003a, 2003b). Therefore, it is not surprising that there are no consistent differences in the relative sizes of the olfactory bulbs between platyrrhines and catarrhines (Fig. 2). It appears that dietary preferences (perhaps the importance of fruits with high levels of sugar) may be an important selective determinant of olfactory abilities (Laska and Hudson, 1993b; Laska, 2001). Activity pattern also plays an important, perhaps even a preponderant, selective role in shaping the main olfactory system. Behavioral observations on nocturnal Aotus (an outlier in having a proportionally larger olfactory bulb than other platyrrhines) suggest this species relies more heavily on olfactory cues for foraging than its diurnal platyrrhine relatives (Bolen and Green, 1997; Bicca-Marques and Garber, 2004).
Whereas anatomical evidence (the size of MOB) appears to agree with expectations from available behavioral evidence, genetic data, as currently interpreted, are discrepant. Based on percentages of intact genes, it has been inferred that there is little or no difference in the size of the intact olfactory receptor gene repertoire between platyrrhines and strepsirrhines but large differences between platyrrhines and catarrhines (Gilad et al., 2003, 2004). Likewise, the data of Roquier et al. (2000) appear to indicate that strepsirrhines and catarrhines have higher percentages of pseudogenes than platyrrhines. We offer several ideas as to why the genetic evidence may be misleading, including uncertainty about the absolute number of OR genes in each species and a lack of knowledge about how (and whether) intact OR genes are expressed as functional receptors. Moreover, single intact OR gene must code for many copies of a particular OR and this is likely an important variable in olfactory ability.
Before commenting on the significance of our findings for fossils, we should make one caveat: neither osteological measurement nor volumetrically determined gross size of the olfactory bulbs of living species can distinguish relative or absolute sizes of the main versus accessory olfactory bulbs. This is because AOB, while recognized functionally and histologically, cannot be visualized as a discrete structure distinct from the olfactory bulb on the external surface of the brain. What is commonly referred to as the olfactory bulb, and its manifestation of the internal surface of the braincase, includes volumetrically both the main and the accessory olfactory structures. Moreover, in living platyrrhines and strepsirrhines, the AOB amounts to such a very small percentage of the volume of the gross anatomical olfactory bulb and as a whole the difference between the volumes of MOB versus MOB plus AOB is not apparent in comparisons of total overall bulb volume in platyrrhines versus catarrhines (Stephan et al., 1981, 1984; Baron et al., 1983). To summarize, it is unlikely that there are anatomical features in a fossil that would allow us to distinguish the accessory olfactory bulb from the size of the olfactory bulb as a whole. Moreover, the primate vomeronasal organ is not surrounded by an osseous structure. It leaves no obvious osteological (and therefore fossil) signature. Thus, given the very small contribution of the AOB to the total volume of the olfactory bulb in primates, any measure of the olfactory fossa size is likely tracking MOB size.
Many have proposed that Tremacebus was a nocturnal primate (Rosenberger, 1979; Szalay and Delson, 1979; Martin, 1990). However, our findings about the size of the olfactory fossa complement previous reports about the size of the orbits in suggesting that Tremacebus was diurnal or at least cathemeral. In relative orbit size, Tremacebus overlaps the distributions of extant nocturnal strepsirrhines but it also overlaps the distributions of extant diurnal platyrrhines (Fig. 11). Moreover, its orbits are far smaller than those of nocturnal haplorhines Aotus and Tarsius. The extreme orbital hypertrophy in Aotus and Tarsius stems from the assumption of a nocturnal activity pattern in the absence of a tapetum lucidum (Kirk and Kay, 2004). If so, Tremacebus could have effectively exploited a nocturnal niche only by reevolving a tapetum lucidum. We consider this an unlikely possibility.
The size of the olfactory fossa reveals additional data pertinent to the question of the activity pattern of Tremacebus. We demonstrate that olfactory fossa of Tremacebus harringtoni (and by inference its main olfactory bulb) is proportionally smaller than that of the only nocturnal anthropoid Aotus and is comparable in relative size to that of extant diurnal platyrrhines. This implies that Tremacebus' sense of smell was no different from diurnal platyrrhines but less acute than in nocturnal Aotus, again supporting the inference of a diurnal activity pattern for Tremacebus.
Our results have implications for the antiquity of the nocturnal activity pattern in anthropoids. If 20-millon-year-old diurnal Tremacebus is a relative of Aotus, nocturnality must have evolved subsequent to the Tremacebus-Aotus node. Indeed, the hypothesis of a Tremacebus-Aotus clade should be reevaluated since the only evidence supporting such a clade is the supposed presence of equivalently enlarged orbits in the two (Horovitz, 1999). The next oldest possible sister taxon to Aotus is Aotus dindensis = Mohanamico hershkovitzi (middle Miocene, Colombia). This species probably had enlarged orbits (Setoguchi and Rosenberger, 1987). On the other hand, if Mohanamico hershkovitzi (Luchterhand et al., 1986) and “Aotus dindensis” are conspecific and related to pitheciines as argued by Kay (1990), then possibly nocturnality arose independently twice in platyrrhine evolution: once in the Mohanamico hershkovitzi lineage and a second time independently in the Aotus lineage. In short, much remains to be learned about the origins of nocturnality in platyrrhines.
The authors thank Dr. Jaime Powell, curator of the Rusconi Collection at Museo de Fundación Miguel Lillo, Tucuman, Argentina, for graciously loaning the skull of Tremacebus. They have especially profited from discussions with Mattias Laska and Matt Cartmill. Supported by National Science Foundation grants BCS-0090255 and IIS-0208675 (to R.F.K., J.B.R., and T.B.R.).