SEARCH

SEARCH BY CITATION

Keywords:

  • Aggression;
  • division of labor;
  • motivation;
  • pair bond;
  • reinforcement;
  • response threshold;
  • sociogenomics

Abstract

  1. Top of page
  2. Abstract
  3. Response threshold model
  4. Division of labor in insect colonies, the response threshold model and the foraging gene
  5. Models of motivation
  6. State space models of behavior
  7. Aggression in decapod crustaceans, state space models and biogenic amines
  8. Models of reward, motivation and reinforcement
  9. Social foraging in bees, reinforcement and biogenic amines
  10. Pair bonding in voles, reinforcement and neuropeptide receptors
  11. Summary
  12. References
  13. Acknowledgments

It is extremely difficult to trace the causal pathway relating gene products or molecular pathways to the expression of behavior. This is especially true for social behavior, which being dependent on interactions and communication between individuals is even further removed from molecular-level events. In this review, we discuss how behavioral models can aid molecular analyses of social behavior. Various models of behavior exist, each of which suggest strategies to dissect complex behavior into simpler behavioral ‘modules.’ The resulting modules are easier to relate to neural processes and thus suggest hypotheses for neural and molecular function. Here we discuss how three different models of behavior have facilitated understanding the molecular bases of aspects of social behavior. We discuss the response threshold model and two different approaches to modeling motivation, the state space model and models of reinforcement and reward processing. The examples we have chosen illustrate how models can generate testable hypotheses for neural and molecular function and also how molecular analyses probe the validity of a model of behavior. We do not champion one model over another; rather, our examples illustrate how modeling and molecular analyses can be synergistic in exploring the molecular bases of social behavior.

New technologies for molecular and genomic analyses and the near exponential growth in the availability of genomic sequence information have contributed to a great increase in the discovery of genes and molecules associated with forms of social behavior (Robinson et al. 2005). Identifying these molecules is one challenge; analyzing how they influence social behavior is another.

The path linking genes or molecules to the expression of behavior is long and complex (Hall 2003). There are innumerable ways genes can influence development, physiology and the nervous system to affect behavior. Further, the genome has a dynamic relationship with behavior, and each influences the other through complex regulatory mechanisms (Robinson 2004). Social behavior adds an additional tier of complexity because social behavior depends on interaction and communication between individuals and is therefore even further removed from molecular changes within an individual.

In this review, we discuss how behavioral models can help relate gene function to social behavior. Models of behavior suggest strategies for dissecting complex behavior into simpler behavioral modules, which are easier to relate to neurobiological processes, and can suggest testable hypotheses for neural and molecular function. The concept of modularity is widespread in fields of biology dealing with complex interacting systems, especially evolutionary and developmental biology. While the concept of modularity is somewhat intuitive, modules of a complex system can be difficult to precisely define. In developmental biology, an assumption of modularity is that a developing organism can be divided into distinct organizational or functional units and these can be described as modules (Carroll 2005). Here, we borrow that perspective from developmental biology to describe a behavioral module as a distinct organizational or functional unit in the expression of complex behavior.

We consider three different models of behavior that have been employed in four case studies exploring the molecular basis of social behavior. First, we discuss how the response threshold model has facilitated molecular analyses of division of labor in honeybees. Then we treat two different modeling approaches to motivation, state space models and models of reward processing and reinforcement, and consider how these models have contributed to understanding the molecular bases of aggression in lobsters, pair bonding in voles and social foraging in honeybees.

Each case study began with a known correlation between a molecular pathway and the expression of social behavior and then explored the function of the molecules in behavior by employing behavioral models to simplify the social behavior under investigation. Our case studies highlight the utility of behavioral models and also the synergy between modeling and molecular analyses, as often molecular analyses refine a model of behavior. This is by no means an exhaustive list of possible modeling approaches, and we do not champion one model over another. Each model has its own assumptions and limitations, and our examples discuss how different models can be useful in different systems.

Response threshold model

  1. Top of page
  2. Abstract
  3. Response threshold model
  4. Division of labor in insect colonies, the response threshold model and the foraging gene
  5. Models of motivation
  6. State space models of behavior
  7. Aggression in decapod crustaceans, state space models and biogenic amines
  8. Models of reward, motivation and reinforcement
  9. Social foraging in bees, reinforcement and biogenic amines
  10. Pair bonding in voles, reinforcement and neuropeptide receptors
  11. Summary
  12. References
  13. Acknowledgments

The response threshold model is one of the oldest and simplest models of animal behavior, and it has proved extremely useful for understanding the behavior of both solitary and social organisms. The model states that responses to various stimuli are based on stimulus thresholds; subthreshold stimuli result in no response, whereas superthreshold stimuli elicit a reaction (Beshers et al. 1999; Bonabeau & Theraulaz 1999; Manning 1967; Page & Erber 2002). The response threshold model draws a parallel between behavioral responses and the responses of neurons. An individual neuron only responds when a stimulus sufficiently depolarizes the membrane potential, eliciting either graded or all-or-nothing responses (depending on the type of neuron).

The response threshold model focuses attention on behavioral responses to key stimuli, suggesting a strategy for dissecting complex traits into relevant modules by considering behavioral responses to each of these stimuli. Further, focusing on each of these modules simplifies molecular analyses by directing attention to pathways that change responsiveness to the key stimuli.

Responsiveness is a catch-all term describing the probability or intensity with which an organism responds to a given stimulus intensity. Its use accepts that a change in behavioral response can be caused by a change in sensitivity to a stimulus, a change in processing of a stimulus or a change in motor response to a stimulus. Further electrophysiological, neuroanatomical or behavioral studies can help to pinpoint the mechanisms involved.

The response threshold concept has long been employed in analyses of division of labor in social insect colonies (Robinson 1987). If the response threshold model is an accurate description of how animals respond to stimuli, then a division of labor may be an inescapable property of co-operative group living (Fewell & Page 1999; Page & Erber 2002; Page & Mitchell 1998). This is because of the negative feedback that typically exists between performing a task and the stimulus level that triggered a response (Robinson & Page 1989a). For example, honeybees (Apis mellifera) and bumble bees (Bombus terrestris) actively thermoregulate their colony to maintain a stable internal temperature by fanning their wings when the colony is too warm to cool it. There is a division of labor in that only a proportion of bees participate in wing fanning. According to the response threshold model, wing fanning is a behavioral response released by a superthreshold temperature. Normally, wing fanning would lower hive temperature to a subthreshold level, at which point fanning would cease (Page & Erber 2002; Weidenmuller et al. 2002).

Division of labor can occur even if all members of a group have the same response thresholds to a stimulus (Bonabeau & Theraulaz 1999; Bonabeau et al. 1997). However, variation in response thresholds can improve colony homeostasis (Jones et al. 2004) and also create ‘specialists’ (Fig. 1) because the workers with the lowest response threshold for a stimulus will be the first to respond and the longest to act (Beshers et al. 1999; Page & Mitchell 1998; Page & Robinson 1991; Weidenmuller 2004).

image

Figure 1. Hypothetical response threshold model to explain interindividual differences in task specialization in an insect colony. Shown is a hypothetical distribution describing the variation in response thresholds to a task-related stimulus between individuals in a colony. Individuals with response thresholds lower than the current stimulus level (shaded area) will be actively engaged in the associated task. Individuals at the margins of the distribution will show ‘extreme’ behavior. Those with very high response thresholds will rarely or never perform the behavior, but bees with very low response thresholds will be most likely to perform the task, and may be classed as ‘specialists’. Adapted from Robinson and Page (1989b).

Download figure to PowerPoint

Division of labor in insect colonies, the response threshold model and the foraging gene

  1. Top of page
  2. Abstract
  3. Response threshold model
  4. Division of labor in insect colonies, the response threshold model and the foraging gene
  5. Models of motivation
  6. State space models of behavior
  7. Aggression in decapod crustaceans, state space models and biogenic amines
  8. Models of reward, motivation and reinforcement
  9. Social foraging in bees, reinforcement and biogenic amines
  10. Pair bonding in voles, reinforcement and neuropeptide receptors
  11. Summary
  12. References
  13. Acknowledgments

Workers in a honeybee colony specialize on different tasks and change roles as they age so that the colony division of labor is structured by a process of behavioral development occurring within individual bees. In a typical colony, an adult worker performs in-hive tasks such as brood care (‘nursing’) when young and shifts to foraging when about 2- to 3-weeks old. Comparative analysis and the candidate gene approach (Fitzpatrick et al. 2005) were used to identify one of the genes involved in regulating the transition from working in the hive to foraging. The foraging gene in Drosophila melanogaster (for) encodes a guanosine 3′,5′-monophosphate (cGMP)-dependent protein kinase (PKG) (Osborne et al. 1997). Allelic variation in for influences variation in larval feeding behavior in Drosophila, causing some to roam widely while foraging (rovers) and others to explore just the immediate area (sitters) (Sokolowski et al. 1997). There is an appealing analogy between rover and sitter Drosophila larvae and the exploratory behavior of honeybee foragers vs. the stay-at-home behavior of nurses. Forager honeybees, like rover flies, have higher expression of the orthologous Amfor in the brain relative to nurses (Ben-Shahar et al. 2002). Further, oral treatment with cGMP causes an increase in brain PKG activity and a precocious onset of foraging (Ben-Shahar et al. 2002).

This evidence suggests an increase in expression of the foraging gene is involved in the onset of foraging behavior in honeybees, but what could PKG be doing to cause a shift from working within the hive to foraging outside the hive? As is often the case in molecular analyses of behavior, the gene of interest (here PKG) is widely expressed and performs numerous roles in the nervous system (Ruth 1999; Stansberry et al. 2001; Wang & Robinson 1997).

Ben-Shahar et al. (2003) employed the response threshold model to dissect foraging into simpler modules, each a response to a key stimulus. In honeybees, the foraging gene is strongly expressed in the visual system, particularly the optic lobe lamina and regions of the mushroom bodies known to receive visual input (Ben-Shahar et al. 2003; Ehmer & Gronenberg 2002; Gronenberg 2001). Therefore Ben-Shahar et al. (2003) examined the effects of PKG on phototactic responses to light. Honeybees experience a major change in exposure to light when they shift from working in the dark hive to foraging outside. Foragers are positively phototactic, much more so than nurses (Ben-Shahar et al. 2003; Menzel & Greggers 1985). Ben-Shahar hypothesized elevated brain PKG increased responsiveness to light and thereby influenced the onset of foraging. In a laboratory bioassay, cGMP treatment increased the phototacic response of young bees (Ben-Shahar et al. 2003). Electroretinogram analysis indicated that the cGMP-induced increase in positive phototaxis was not based on effects of sensitivity to light per se. Perhaps PKG is involved in modifying the function of neuronal circuits in the lamina and/or mushroom bodies via phosphorylation of some component molecules, which is similar to the affect of PKG on olfaction in mammals (Kroner et al. 1996).

In this example, the response threshold model provided a framework for dissecting a social dynamic (division of labor) into simpler modules. Tissue localization of the expression changes in Amfor suggested which modules might be relevant, and these inferences together resulted in a testable hypothesis for the role of Amfor in honeybee division of labor. There likely are other ways in which PKG influences foraging behavior, but by focusing on one module (light responsiveness) progress has been made toward understanding the full mechanism.

The response threshold model is valuable because it suggests a clear stimulus-specific dissection of behavior and can so easily be related to different levels of biological organization (group, individual or neuron). However, a simple response threshold model cannot address how behavior might depend on interactions between stimuli or how responses to stimuli could qualitatively change. In these instances, more complex behavioral models are needed.

Models of motivation

  1. Top of page
  2. Abstract
  3. Response threshold model
  4. Division of labor in insect colonies, the response threshold model and the foraging gene
  5. Models of motivation
  6. State space models of behavior
  7. Aggression in decapod crustaceans, state space models and biogenic amines
  8. Models of reward, motivation and reinforcement
  9. Social foraging in bees, reinforcement and biogenic amines
  10. Pair bonding in voles, reinforcement and neuropeptide receptors
  11. Summary
  12. References
  13. Acknowledgments

In its simplest form, the response threshold model assumes that response thresholds are fixed, but animals do not always respond in the same way to a given stimulus. For example, animals sometimes initiate food-searching behavior and lobsters will sometimes initiate aggressive encounters, while at other times they avoid them (Huber et al. 1997). Alterations in the responsiveness of an individual can be ascribed to changes in motivation. There is a certain arbitrariness to the definition of what constitutes a change in motivation (Hinde 1982), and more than one behavioral phenomenon falls under this umbrella. Consequently, there are many different approaches to modeling motivational phenomena. Here we focus on two, state space models and models of reinforcement, and we consider how these models have aided molecular analyses of aggression in lobsters, social foraging in honeybees and pair bonding in voles.

State space models of behavior

  1. Top of page
  2. Abstract
  3. Response threshold model
  4. Division of labor in insect colonies, the response threshold model and the foraging gene
  5. Models of motivation
  6. State space models of behavior
  7. Aggression in decapod crustaceans, state space models and biogenic amines
  8. Models of reward, motivation and reinforcement
  9. Social foraging in bees, reinforcement and biogenic amines
  10. Pair bonding in voles, reinforcement and neuropeptide receptors
  11. Summary
  12. References
  13. Acknowledgments

The response threshold model considers a single stimulus and predicts whether the stimulus intensity is sufficient to elicit a behavioral response or not. But in the real world, animals are exposed to a gestalt of stimuli and must ‘choose’ the most appropriate response from many possible options. State space models propose a multivariate solution to model such complex ‘decisions’ by considering how responses to different stimuli might interact (McFarland & Houston 1981; McFarland & Sibly 1975).

State space models postulate that the total behavioral repertoire is controlled by a set of causal factors (McFarland & Sibly 1975). External factors, such as the perception of food stimuli, and internal ‘motivational’ factors, such as the size of fat stores, circadian rhythms or reproductive condition, are all considered causal factors and treated similarly in the model. The state of these causal factors can be represented as a Euclidean space, the axes of which represent the causal factors. This causal factor space is multidimensional, with the number of dimensions depending on the number of causal factors relevant to the animal’s behavior.

McFarland and Sibly (1975) assume that an animal’s behavioral repertoire can be classified into mutually exclusive categories (activities) such that actions belonging to one activity are incompatible with actions belonging in another. For example, all the actions associated with fighting can be considered one activity and all the actions associated with mating another because an animal cannot fight and mate at the same time.

The modeling process, summarized by McFarland and Houston (1981), maps behavioral tendencies onto the causal factor space such that every point in the causal factor space is associated with a tendency of performing a specific action. A consequence of this model is that there can be more than one point in causal factor space with the same behavioral tendency. For example, assuming for simplicity that feeding depends solely on the two causal factors, food availability and hunger, the same feeding tendency might be observed in a situation of high food availability and low hunger as when there is low food availability and high hunger. Points in causal factor space sharing the same behavioral tendencies are linked by motivational isoclines (Fig. 2).

image

Figure 2. The state space model: hypothetical causal factor space for feeding. This model assumes that feeding tendency is influenced by hunger and food cues only. The feeding tendency is the same at points X1, …, X5, and the solid line joining these points is a motivational isocline. Point A represents the causal factor state of an animal before feeding and point B after feeding (assuming that feeding reduces both hunger and local food availability). The dashed arrow indicates the trajectory in causal factor space that results from feeding. Adapted from McFarland and Houston (1981).

Download figure to PowerPoint

The causal factor state is a point in causal factor space that describes the animal’s current state. The causal factor state is constantly changing because of shifts in environmental conditions, internal factors and the outcomes of the animal’s own behavior, and this change can be represented as a trajectory in causal factor space. As the trajectory crosses motivational isoclines, there will be a shift in the relative balance of behavioral tendencies. If the causal factor state moves to an area of causal factor space where a different behavioral tendency is dominant, a change in behavior will be observed (Fig. 2).

As an example, in the carnivorous opisthobranch snail Pleurobranchaea californica, food stimuli can induce both avoidance responses and feeding responses. The behavioral tendency depends on both hunger state and feeding stimuli. In a satiated animal, low concentration feeding stimuli trigger avoidance responses, but in a hungry animal, the same concentration of feeding stimuli can trigger feeding responses (Gillette et al. 2000). In the terms of the state space model, food stimuli and hunger are causal factors related to both avoidance behavior and feeding behavior. If feeding stimuli are fixed at a low level but hunger increases, then the causal factor state shifts into a region of causal factor space where the feeding behavioral tendency is dominant.

The state space approach, therefore, makes it possible to predict when a change in conditions will result in a change in behavior. State space models demystify motivation by dissecting the internal state into a series of causal factors, which can be quantified and fed into the model in the same way as external factors. The result is a program for control of behavior that depends on the sum of all causal factors. The philosophies behind response threshold models and state space models are similar in that both assume behavior is released when causal stimulus intensity exceeds threshold, but the multidimensional structure of state space models also allows them to consider how different causal factors (external or internal) can interact. This gives the state space model an enormous advantage over the response threshold model when considering complex systems. State space models may be more versatile, but they require a far greater knowledge of the behavioral system than response threshold models.

Aggression in decapod crustaceans, state space models and biogenic amines

  1. Top of page
  2. Abstract
  3. Response threshold model
  4. Division of labor in insect colonies, the response threshold model and the foraging gene
  5. Models of motivation
  6. State space models of behavior
  7. Aggression in decapod crustaceans, state space models and biogenic amines
  8. Models of reward, motivation and reinforcement
  9. Social foraging in bees, reinforcement and biogenic amines
  10. Pair bonding in voles, reinforcement and neuropeptide receptors
  11. Summary
  12. References
  13. Acknowledgments

With the exception of mating and courtship, most intraspecific encounters in lobsters and crayfish are agonistic, with fights escalating until one individual withdraws. The biogenic amines octopamine and serotonin were first implicated in agonistic behavior by the observations that acute injection of serotonin and octopamine into freely moving lobsters (Homarus americanus) generated postures resembling those seen in dominant (serotonin like) and subordinate (octopamine like) animals when they encounter one another (Huber et al. 1997). Later studies showed that infusion of serotonin in crayfish (Astacus astacus) caused treated individuals to engage larger opponents in prolonged bouts of fighting, even in instances that carried substantial risk of injury (Kravitz & Huber 2003). These analyses identified a neurochemical pathway involved in decapod aggression, but did not explain how changing serotonergic signaling impacted on decapod behavior to make the animals more aggressive.

Huber et al. (1997) proposed that serotonin changed the responsiveness of decapods to some aspects of conspecifics to increase the likelihood and duration of agonistic encounters, but the response threshold model was insufficient to embrace the full complexity of decapod fighting behavior (Huber & Delago 1998). The expression of different fighting strategies varies with motivational causal factors, such as hunger states and previous agonistic success, and external causal factors, such as proximity to a shelter or food resource (Kravitz & Huber 2003). The progress of the fight and its eventual outcome are determined both by an animal’s prior actions in the fight and its responses to the opponent (Kravitz & Huber 2003). Winners are much more likely to dominate future encounters, whereas losers are much more likely to retreat. Consequently, fights are influenced by a web of interdependent causal factors with any action effecting subsequent actions and the responses from the opponent.

Huber and Delago (1998) used detailed ethological dissection and a multivariate principal components analysis to determine the effects of serotonin on the progression of fights. This analysis embraced a state space approach to motivation by considering how the many causal factors that influence a fight sequence were interrelated. Their results showed that the dominant effect of serotonin was to change the decision to retreat, decreasing the likelihood that subordinates would withdraw from the attacks of opponents (Huber & Delago 1998; Huber et al. 1997). In the terms of the state space model, experimentally elevating serotonin moved lobsters to an area of causal factor space where the tendency to retreat was very low.

Fighting history interacts with serotonin level to influence aggression levels. Prior winners are more likely to dominate in fights and prior losers are more likely to retreat, so that in terms of the state space model prior winners with high serotonin levels are in a causal factor state where aggressive behavioral tendencies dominate. Electrophysiological and pharmacological studies then focused on how serotonin could interact with fighting history. Serotonin affected the excitability of peripheral lateral giant (LG) neurons differently in dominant and subordinate individuals, increasing the excitability of the neurons in dominant individuals and decreasing it in subordinates (Yeh et al. 1997). Pharmacological studies suggest that the change in response to serotonin is caused by a long-term change in activity of 5HT1 and 5HT2 receptor classes on the LG neurons (Yeh et al. 1996, 1997). LG neurons are part of a reflexive escape circuit that is activated by sudden taps to the tail and is not under voluntary control. Activation of this escape circuit causes a powerful tail flick that propels an animal forward. During a fight, activation of LG would involuntarily pitch an animal toward its opponent, which could be disastrous for a subordinate animal trying to make a retreat; therefore, it would be adaptive to inhibit this reflex response in subordinate animals (Yeh et al. 1997).

The next step in unraveling serotonin’s role in aggression is to determine what stimuli release serotonin to activate the 5HT1 and 5HT2 receptors. Behavioral analyses suggest that serotonin is released by threatening stimuli (Huber & Delago 1998; Huber et al. 1997), but the nature of these stimuli have yet to be explored. In this example, the state space approach enabled a dissection of decapod fight behavior into simpler modules that proposed hypotheses for serotonin’s role in modulating retreat.

Models of reward, motivation and reinforcement

  1. Top of page
  2. Abstract
  3. Response threshold model
  4. Division of labor in insect colonies, the response threshold model and the foraging gene
  5. Models of motivation
  6. State space models of behavior
  7. Aggression in decapod crustaceans, state space models and biogenic amines
  8. Models of reward, motivation and reinforcement
  9. Social foraging in bees, reinforcement and biogenic amines
  10. Pair bonding in voles, reinforcement and neuropeptide receptors
  11. Summary
  12. References
  13. Acknowledgments

State space models map behavioral tendencies onto causal factor space to predict what an animal will do in a given state. The state space model treats all causal factor states equally and makes no judgments about the ‘value’ of different states; however, a fundamentally different approach to modeling motivation begins with the view that some causal factor states are preferable to others. It is often assumed that animal behavior is organized such that causal factor states conducive to survival and reproduction are valued over states conducive to harm (Manning & Dawkins 1992). Valuation mechanisms are assumed to be hardwired into the nervous system so that animals prefer certain states over others and aim to achieve a more desirable state. Assigning states an internal value yields a system for ascribing desires or goals to animal behavior and presents an alternative system for predicting behavior. In this context, motivation is often used to describe the internal ‘drive’ directing animal behavior toward a goal or a preferred state.

Computational models have been developed to describe how the vertebrate nervous system sets goals, computes values of particular resources or options and uses both to guide sequences of behavioral choices. These models have developed hand in hand with neurophysiological analyses of reward-processing mechanisms. Reinforcement learning (a branch of computational theory) has been particularly influential in designing and interpreting experiments that probe reward processing in the vertebrate brain (Montague et al. 2004).

Reinforcement learning theory models how animals learn to achieve a desired state and avoid undesirable ones as efficiently as possible. States engendered by rewards are referred to as goals. In reinforcement learning, it is often assumed that a fundamental goal is to take actions that are most likely to lead to the greatest rewards in the future. This strategy is achieved under the guidance of reinforcement signals, which serve to ‘criticize’ actions with respect to how well they serve the ultimate goal (Montague et al. 2004). In temporal difference learning, the critical information is the reward prediction error (or temporal-difference error). The reward prediction error is a function that compares the received reward with the predicted reward and also incorporates information about the next prediction made by the reward prediction system (Dayan & Abbott 2001; Doya 2002; Sutton 1988).

Reinforcement learning models are highly abstract, but there is growing evidence that neural algorithms in the mammalian brain process reward using principles similar to those described in such models. Schultz and colleagues (Hollerman & Schultz 1998; Schultz 2001, 2002; Schultz et al. 1993) have shown that, in monkeys, dopaminergic neurons in the ventral tegmental area (VTA) and substantia nigra show phasic changes in spike activity that correlate with the history of reward delivery. These changes seem to encode the prediction error about a future reward. Bursts of activity in these neurons signal a positive reward prediction error (things are better than expected), while pauses in firing signal a negative prediction error (worse than expected). Baseline firing activity signals when the reward prediction matched the outcome (Hollerman & Schultz 1998). Berridge and Robinson (1998) argue that dopamine release from these neurons into the VTA and substantia nigra mediates the attachment of reward value to an action or an object to influence future behavioral choices.

If we assume that animals are motivated to maximize future rewards and can learn the rewarding properties of various stimuli, then reinforcement theory becomes a powerful method for predicting and analyzing diverse aspects of behavior.

Social foraging in bees, reinforcement and biogenic amines

  1. Top of page
  2. Abstract
  3. Response threshold model
  4. Division of labor in insect colonies, the response threshold model and the foraging gene
  5. Models of motivation
  6. State space models of behavior
  7. Aggression in decapod crustaceans, state space models and biogenic amines
  8. Models of reward, motivation and reinforcement
  9. Social foraging in bees, reinforcement and biogenic amines
  10. Pair bonding in voles, reinforcement and neuropeptide receptors
  11. Summary
  12. References
  13. Acknowledgments

Reinforcement models have been developed with mammals in mind: can they also be applied to invertebrates? Certainly, there are many aspects of invertebrate behavior that appear goal directed and could be interpreted in terms of reinforcement models. An extreme example is social foraging in honeybees. Forager honeybees ceaselessly gather floral resources for their colony. Foraging certainly appears goal directed as foragers travel large distances to seek out floral resources, and it appears sensitive to reinforcement as foragers rapidly adapt their behavior to optimize the energetic efficiency of their foraging trips (Seeley 1995, 1998). The behavior of foragers is also extremely sensitive to the state of the colony. Foragers assess the colony’s pollen stocks, and complex communication systems exist between foragers and hive bees so that the influx of pollen and nectar is in step with the colony’s needs (Seeley 1995). The most impressive communication system is the honeybee waggle dance language, which describes the ritualized symbolic dances performed by foragers on return to the hive to communicate the location and desirability of floral resources to nest mates (Dyer 2002). Dances recruit additional foragers to gather the resource. Honeybee social foraging can be viewed as a goal-directed behavior, with foragers operating under guidance of complex reinforcement signals and driven to optimize the condition of the colony. Adopting this approach facilitated analysis of how the neurochemical octopamine acts to modulate honeybee dance communication.

A high level of octopamine in the antennal lobes of the honeybee brain is associated with foraging behavior in honeybees, and octopamine treatments stimulate foraging (Schulz et al. 2002). Initial investigations applied the response threshold model to dissect division of labor and suggested that octopamine operated to increase responsiveness to brood pheromone, a key pheromonal stimulator of foraging (Barron et al. 2002), but further studies showed the story to be more complex. Octopamine did not simply increase responsiveness to brood pheromone. Brood pheromone is a complex signal involved in brood care as well as foraging, and octopamine caused a shift in how bees responded to brood pheromone, increasing the foraging response and decreasing a brood care response (Barron & Robinson 2005).

Octopamine not only stimulates foraging, but also modulates dance behavior. Octopamine treatments stimulated returning foragers to dance and selectively increased the reporting of resource value in dances by forager bees (Barron et al. 2007). Octopamine treatment did not simply release dancing in all bees; rather octopamine treatment caused a dose-dependent increase in the likelihood of returning foragers dancing. Closer analysis of the structure of the dance showed that while octopamine-treated bees communicated the location of resources accurately, their dances were longer and more vigorous than those of untreated bees. Dance duration and vigor both communicate desirability of the resource for the colony, and therefore octopamine treatment caused bees to exaggerate resource value in their dances (Barron et al. 2007). Together, these findings indicated that the role of octopamine in social foraging was too complex to be considered under the response threshold model.

The roles of octopamine in modulating reward-seeking activity and the reporting of resource value via dances suggest that octopamine could be involved in neural mechanisms of reward valuation and reinforcement in the bee brain. This hypothesis is strongly supported by the role octopamine also plays in learning and memory in both honeybees (Menzel 2001; Menzel & Giurfa 2001) and D. melanogaster (Schwaerzel et al. 2003). In Drosophila, octopaminergic neurons are necessary for fruit flies to learn rewarding stimuli (Schwaerzel et al. 2003). In honeybees, a microinjection of octopamine into the mushroom bodies or antennal lobes of the bee brain was an effective substitute for a sucrose reward in an associative learning paradigm training bees to associate sucrose with an odor (Hammer & Menzel 1998). This evidence suggests octopamine mediates the learning of rewarding stimuli and may be the neurochemical released by perception of sucrose that represents the reinforcing properties of rewards in the insect brain. The role of octopamine in positive reinforcement in insects seems to parallel the role of dopamine in reinforcement in the mammalian brain (Wise 2004; Wise & Rompre 1989).

In this example, the stimulus response model was rejected as too simple a model to embrace the multiple roles of octopamine in honeybee social foraging. The multiple roles of octopamine in reward learning, social foraging and dance communication are consistent with octopamine being part of the neural mechanism of reinforcement in the bee brain. Considering social foraging in terms of reinforcement will suggest new hypotheses for exploring the cellular and circuit basis of motivation and drive in insects, particularly because a foraging bee is not working for self-interest but rather working to improve the state of the colony. Octopamine is involved in feeding and food seeking in solitary insects, such as blow flies, crickets and Drosophila (Long & Murdock 1983; Schwaerzel et al. 2003; Unoki et al. 2005), and social foraging in honeybees. Comparing octopaminergic circuitry in social and related solitary insects might shed some light on how molecular pathways that regulate hunger and feeding in solitary organisms have been shaped by social evolution to regulate social foraging.

Pair bonding in voles, reinforcement and neuropeptide receptors

  1. Top of page
  2. Abstract
  3. Response threshold model
  4. Division of labor in insect colonies, the response threshold model and the foraging gene
  5. Models of motivation
  6. State space models of behavior
  7. Aggression in decapod crustaceans, state space models and biogenic amines
  8. Models of reward, motivation and reinforcement
  9. Social foraging in bees, reinforcement and biogenic amines
  10. Pair bonding in voles, reinforcement and neuropeptide receptors
  11. Summary
  12. References
  13. Acknowledgments

Recent studies with voles have employed hypotheses drawn from reinforcement models to explore the molecular basis of pair bond formation. Closely related species of vole display strongly contrasting social organizations. Prairie voles (Microtus ochrogaster) are usually monogamous, but the montane vole (Microtus montanus) and meadow vole (Microtus pennsylvanicus) are nonmonogamous (Insel et al. 1995). The neuropeptides oxytocin and vasopressin are known to be involved in social affiliation in mammals (Insel 1992), prompting a comparison of the oxytocin and vasopressin systems in montane and prairie voles. Compared with nonmonogamous species, monogamous female prairie voles have higher densities of oxytocin receptors (OTRs) in the nucleus accumbens, and males have higher densities of vasopressin receptor 1a (V1aR) in the ventral pallidum and medial amygdala (Insel & Young 2001). Furthermore, experimental overexpression of V1aR in the nonmonogamous male meadow vole was sufficient to enhance pair bonding (Lim et al. 2004).

How do V1aR and OTR influence pair bonding? The first clue came from the observation that V1aR and OTR are both expressed in brain regions commonly associated with the vertebrate dopaminergic reward-processing system (Young & Wang 2004; Young et al. 2001). A release of dopamine into the ventral pallidum and nucleus accumbens mediates the attachment of reward value to an action or an object to influence future behavioral choices (Wise 2004). Insel and Young (2001) hypothesized that if oxytocin and vasopressin activated brain regions sensitive to reward and reinforcement, pair bond formation could involve a form of reinforcement. They suggested that for monogamous species, features of the partner are strongly associated with an innately reinforcing stimulus, mating, so that partners are motivated to remain together.

Molecular studies support this interpretation of pair bonding. Both oxytocin and vasopressin are involved in the neural processing of sensory cues involved in social learning, especially learning the olfactory signatures of conspecifics (Ferguson et al. 2000). Mating causes a release of vasopressin and oxytocin into the prefrontal cortex, nucleus accumbens and ventral pallidum (Young & Wang 2004). Mating in both males and females also causes dopamine release from the VTA into the prefrontal cortex and nucleus accumbens (Young & Wang 2004). Concurrent activation of the oxytocin/vasopressin and the dopamine pathway is necessary for pair bond formation (Fig. 3). In nonmonogamous species, the lower density of OTR and V1aR in the nucleus accumbens and ventral pallidum suggests that the dopamine system and oxytocin/vasopressin systems are only weakly coupled, so the smell of a partner might not be intimately associated with the rewarding mating experience (Young & Wang 2004).

image

Figure 3. Neurobiological model for pair bonding in male monogamous prairie voles based on reward theory. The act of mating stimulates the VTA, which releases dopamine into the nucleus accumbens and prefrontal cortex. Olfactory cues from the female are processed by the olfactory bulb and the medial amygdala (AM), which are critical for social recognition. The medial AM releases vasopressin to the ventral pallidum. Concurrent activation of dopamine D2 receptors in the nucleus accumbens and vasopressin V1aR receptors in the ventral pallidum of males results in a conditioned pair bond.

Download figure to PowerPoint

The V1aR is coded for by the arginine vasopressin receptor 1a gene (avpr1a). Differences in expression pattern of the V1aR between the two vole species do not seem to be because of sequence differences in the coding region of avpr1a; rather, expression differences seem due to differences in the length of a microsatellite sequence in the regulatory region of the gene (Young & Wang 2004). In prairie voles, polymorphism of the microsatellite in the avpr1a regulatory region can even account for individual differences in social affiliation behavior (Hammock & Young 2005; Hammock et al. 2005).

In this example, molecular analyses first suggested a link between pair bonding and models of reinforcement, and the hypotheses generated by applying a reinforcement model to pair bonding have proven extremely useful in exploring the molecular basis of the pair bond.

Summary

  1. Top of page
  2. Abstract
  3. Response threshold model
  4. Division of labor in insect colonies, the response threshold model and the foraging gene
  5. Models of motivation
  6. State space models of behavior
  7. Aggression in decapod crustaceans, state space models and biogenic amines
  8. Models of reward, motivation and reinforcement
  9. Social foraging in bees, reinforcement and biogenic amines
  10. Pair bonding in voles, reinforcement and neuropeptide receptors
  11. Summary
  12. References
  13. Acknowledgments

This review has discussed cases that illustrate the synergy between modeling and molecular analyses of behavior. No model of behavior is necessarily better than another; rather, different models are useful in different contexts. In the examples described above, the value of modeling is not to produce a perfect model of behavior, rather molecular and modeling analyses of behavior can progress together to understand how complex behavior is controlled. Models can aid in the dissection of complex traits into simpler modules and suggest hypotheses for testing how identified molecular pathways might influence behavior. This was illustrated by the application of the response threshold model to division of labor in bees to explore the role of the foraging gene in division of labor and honeybee behavioral development. Conversely, molecular analyses can improve models of behavior. Initial molecular analyses of pair bonding in mammals implied a link between pair bond formation and reward processing. Subsequent modeling of pair bonding according to reinforcement theory has suggested novel hypotheses for the control of social affiliation.

We have entered an exciting phase in the investigation of social behavior with the increased availability of new genomic and molecular techniques accelerating enormously the identification of candidate molecules for aspects of social behavior. A dual approach considering both modeling and molecular analyses is likely to be most effective in solving how molecular pathways influence social behavior.

References

  1. Top of page
  2. Abstract
  3. Response threshold model
  4. Division of labor in insect colonies, the response threshold model and the foraging gene
  5. Models of motivation
  6. State space models of behavior
  7. Aggression in decapod crustaceans, state space models and biogenic amines
  8. Models of reward, motivation and reinforcement
  9. Social foraging in bees, reinforcement and biogenic amines
  10. Pair bonding in voles, reinforcement and neuropeptide receptors
  11. Summary
  12. References
  13. Acknowledgments
  • Barron, A.B. & Robinson, G.E. (2005) Selective modulation of task performance by octopamine in honey bee (Apis mellifera) division of labor. J Comp Physiol A 191, 659668.
  • Barron, A.B., Schulz, D.J. & Robinson, G.E. (2002) Octopamine modulates responsiveness to foraging-related stimuli in honey bees (Apis mellifera). J Comp Physiol A 188, 603610.
  • Barron, A.B., Maleszka, R., Meer, R.K.V. & Robinson, G.E. (2007) Octopamine modulates honey bee dance behavior. Proc Natl Acad Sci U S A 104, 17031707.
  • Ben-Shahar, Y., Robichon, A., Sokolowski, M.B. & Robinson, G.E. (2002) Influence of gene action across different time scales on behavior. Science 296, 741744.
  • Ben-Shahar, Y., Leung, H., Pak, W., Sokolowski, M. & Robinson, G. (2003) cGMP-dependent changes in phototaxis: a possible role for the foraging gene in honey bee division of labor. J Exp Biol 59, 269278.
  • Berridge, K.C. & Robinson, T.E. (1998) What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Rev 28, 309369.
  • Beshers, S.N., Robinson, G.E. & Mittenthal, J.E. (1999) Response thresholds and division of labor in insect colonies. In Detrain, C., Deneubourg, J.L. & Pasteels, J.M. (eds), Information Processing in Social Insects. Birkhauser Verlag, Basel, pp. 115139.
  • Bonabeau, E. & Theraulaz, G. (1999) Role and variability of response thresholds in the regulation of division of labor in insect societies. In Detrain, C., Deneubourg, J.L. & Pasteels, J.M. (eds), Information Processing in Social Insects. Birkhäuser, Basel, pp. 141164.
  • Bonabeau, E., Theraulez, G., Deneubourge, J.-L., Aron, S. & Camazine, S. (1997) Self-organization in social insects. Trends Ecol Evol 12, 188193.
  • Carroll, S.B. (2005) Endless Forms Most Beautiful: The New Science of Evo Devo. W.W. Norton, New York.
  • Dayan, P. & Abbott, L.F. (2001) Theoretical Neuroscience. MIT Press, Cambridge, MA.
  • Doya, K. (2002) Metalearning and neuromodulation. Neural Netw 15, 495506.
  • Dyer, F.C. (2002) The biology of the dance language. Ann Rev Entomol 47, 917949.
  • Ehmer, B. & Gronenberg, W. (2002) Segregation of visual input to the mushroom bodies in the honeybee (Apis mellifera). J Comp Neurol 451, 362373.
  • Ferguson, J.N., Young, L.J., Hearn, E.F., Insel, T.R. & Winslow, J.T. (2000) Social amnesia in mice lacking the oxytocin gene. Nat Genet 25, 284288.
  • Fewell, J.H. & Page, R.E. (1999) The emergence of division of labour in forced associations of normally solitary ant queens. Evol Ecol Res 1, 537548.
  • Fitzpatrick, M.J., Ben-Shahar, Y., Smid, H.M., Vet, L.E.M., Robinson, G.E. & Sokolowski, M.B. (2005) Candidate genes for behavioural ecology. Trends Ecol Evol 20, 96104.
  • Gillette, R., Huang, R.-C., Hatcher, N. & Moroz, L.L. (2000) Cost-benefit analysis potential in feeding behavior of a predatory snail by integration of hunger, taste and pain. Proc Natl Acad Sci U S A 97, 35853590.
  • Gronenberg, W. (2001) Subdivisions of hymenopteran mushroom body calyces by their afferent supply. J Comp Neurol 435, 474489.
  • Hall, J.C. (2003) A neurogeneticist’s manifesto. J Neurogenet 17, 190.
  • Hammer, M. & Menzel, R. (1998) Multiple sites of associative odor learning as revealed by local brain microinjections of octopamine in honeybees. Learn Mem 5, 146156.
  • Hammock, E.A.D. & Young, L.J. (2005) Microsatellite instability generates diversity in brain and sociobehavioral traits. Science 308, 16301634.
  • Hammock, E.A.D., Lim, M.M., Nair, H.P. & Young, L.J. (2005) Association of vasopressin 1a receptor levels with a regulatory microsatellite and behavior. Genes Brain Behav 4, 289301.
  • Hinde, R.A. (1982) Ethology. Oxford University Press, Oxford.
  • Hollerman, J.R. & Schultz, W. (1998) Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1, 304309.
  • Huber, R. & Delago, A. (1998) Serotonin alters decisions to withdraw in fighting crayfish, Astacus astacus: the motivational concept revisited. J Comp Physiol A 182, 573583.
  • Huber, R., Smith, K., Delago, A., Isaksson, K. & Kravitz, E.A. (1997) Serotonin and aggressive motivation in crustaceans: altering the decision to retreat. Proc Natl Acad Sci U S A 94, 59395942.
  • Insel, T.R. (1992) Oxytocin – a neuropeptide for affiliation – evidence from behavioral, receptor autoradiographic, and comparative-studies. Psychoneuroendocrinology 17, 335.
  • Insel, T.R. & Young, L.J. (2001) The neurobiology of attachment. Nat Rev Neurosci 2, 129136.
  • Insel, T.R., Preston, S. & Winslow, J.T. (1995) Mating in the monogamous male: behavioral consequences. Physiol Behav 57, 615627.
  • Jones, J.C., Myerscough, M.R., Graham, S. & Oldroyd, B.P. (2004) Honey bee nest thermoregulation: diversity promotes stability. Science 305, 402404.
  • Kravitz, E.A. & Huber, R. (2003) Aggression in invertebrates. Curr Opin Neurobiol 13, 736743.
  • Kroner, C., Boekhoff, I., Lohmann, S.M., Genieser, H.G. & Breer, H. (1996) Regulation of olfactory signalling via cGMP-dependent protein kinase. Euro J Biochem 236, 632637.
  • Lim, M.M., Wang, Z.X., Olazabal, D.E., Ren, X.H., Terwilliger, E.F. & Young, L.J. (2004) Enhanced partner preference in a promiscuous species by manipulating the expression of a single gene. Nature 429, 754757.
  • Long, T.F. & Murdock, L.L. (1983) Stimulation of blowfly feeding behavior by octopamiergic drugs. Proc Natl Acad Sci U S A 80, 41594163.
  • Manning, A. (1967) An Introduction to Animal Behavior. Addison-Wesley, Reading, MA.
  • Manning, A. & Dawkins, M.S. (1992) An Introduction to Animal Behaviour. Cambridge University Press, Cambridge.
  • McFarland, D. & Houston, A. (1981) Quantitative Ethology. Pitman Advanced Publishing Program, Boston.
  • McFarland, D.J. & Sibly, R.M. (1975) The behavioural final common path. Philos Trans R Soc Lond B 270, 265293.
  • Menzel, C.R. & Greggers, U. (1985) Natural phototaxis and its relationship to color vision in honey bees. J Comp Physiol A 157, 311322.
  • Menzel, R. (2001) Searching for the memory trace in a mini-brain, the honeybee. Learn Mem 8, 5362.
  • Menzel, R. & Giurfa, M. (2001) Cognitive architecture of a mini-brain: the honeybee. Trends Cogn Sci 5, 6271.
  • Montague, P.R., Hyman, S.E. & Cohen, J.D. (2004) Computational roles for dopamine in behavioural control. Nature 431, 760767.
  • Osborne, K.A., Robichon, A., Burgess, E., Butland, S., Shaw, R.A., Coulthard, A., Pereira, H.S., Greenspan, R.J. & Sokolowski, M.B. (1997) Natural behavior polymorphism due to a cGMP-dependent protein kinase of Drosophila. Science 277, 834836.
  • Page, R.E. & Erber, J. (2002) Levels of behavioral organization and the evolution of division of labor. Naturwissen 89, 91106.
  • Page, R.E. & Mitchell, S.D. (1998) Self-organisation and the evolution of division of labor. Apidologie 29, 171190.
  • Page, R.E. & Robinson, G.E. (1991) The genetics of division of labour in honey bee colonies. Adv Insect Physiol 23, 117169.
  • Robinson, G.E. (1987) Modulation of alarm pheromone perception in the honey bee: evidence for division of labor based on hormonally regulated response thresholds. J Comp Physiol 160, 613619.
  • Robinson, G.E. (2004) Beyond nature and nurture. Science 304, 397399.
  • Robinson, G.E. & Page, R.E. (1989a) Genetic determination of nectar foraging, pollen foraging, and nest-site scouting in honey bee colonies. Behav Ecol Sociobiol 24, 317323.
  • Robinson, C.E. & Page, R.E. (1989b) The genetic basis of division of labor in an insect society. In Breed, M.D. & Page, R.E. (eds), The Genetics of Social Evolution. Westview Press, Boulder, CO, pp. 6180.
  • Robinson, G.E., Grozinger, C.M. & Whitfield, C.W. (2005) Sociogenomics: social life in molecular terms. Nat Rev Genet 6, 257270.
  • Ruth, P. (1999) Cyclic GMP-dependent protein kinases: understanding in vivo functions by gene targeting. Pharmacol Ther 82, 355372.
  • Schulz, D.J., Barron, A.B. & Robinson, G.E. (2002) A role for octopamine in honey bee division of labor. Brain Behav Evol 60, 350359.
  • Schultz, W. (2001) Reward signaling by dopamine neurons. Neuroscientist 7, 293302.
  • Schultz, W. (2002) Getting formal with dopamine and reward. Neuron 36, 241263.
  • Schultz, W., Apicella, P. & Ljungberg, T. (1993) Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13, 900913.
  • Schwaerzel, M., Monasterioti, M., Scholz, H., Friggi-Grelin, F., Birman, S. & Heisenberg, M. (2003) Dopamine and octopamine differentiate between aversive and appetitive olfactory memories in Drosophila. J Neurosci 23, 1049510502.
  • Seeley, T.D. (1995) The Wisdom of the Hive. Harvard University Press, Cambridge.
  • Seeley, T.D. (1998) Thoughts on information and integration in honey bee colonies. Apidologie 29, 6780.
  • Sokolowski, M.B., Pereira, H.S. & Hughes, K. (1997) Evolution of foraging behavior in Drosophila by density-dependent selection. Proc Natl Acad Sci U S A 94, 73737377.
  • Stansberry, J., Baude, E.J., Taylor, M.K., Chen, P.-J., Jin, S.-W., Ellis, R.E. & Uhler, M.D. (2001) A cGMP-dependent protein kinase is implicated in wild-type motility in C. elegans. J Neurochem 76, 11771187.
  • Sutton, R.S. (1988) Learning to predict by the models of temporal difference. Mach Learn 3, 944.
  • Unoki, S., Matsumoto, Y. & Mizunami, M. (2005) Participation of octopaminergic reward system and dopaminergic punishment system in insect olfactory learning revealed by pharmacological study. Eur J Neurosci 22, 14091416.
  • Wang, X. & Robinson, P.J. (1997) Cyclic GMP-dependent protein kinase and cellular signaling in the nervous system. J Neurochem 68, 443456.
  • Weidenmuller, A. (2004) The control of nest climate in bumblebee (Bombus terrestris) colonies: interindividual variability and self reinforcement in fanning response. Behav Ecol 15, 120128.
  • Weidenmuller, A., Kleineidam, C. & Tautz, J. (2002) Collective control of nest climate parameters in bumblebee colonies. Anim Behav 63, 10651071.
  • Wise, R. & Rompre, P. (1989) Dopamine and reward. Annu Rev Psychol 40, 191225.
  • Wise, R.A. (2004) Dopamine, learning and motivation. Nat Rev Neurosci 5, 483494.
  • Yeh, S.R., Fricke, R.A. & Edwards, D.H. (1996) The effects of social experience on serotonergic modulation of the escape circuit of crayfish. Science 271, 366369.
  • Yeh, S.R., Musolf, B.E. & Edwards, D.H. (1997) Neuronal adaptations to changes in the social dominance status of crayfish. J Neurosci 17, 697708.
  • Young, L.J. & Wang, Z.X. (2004) The neurobiology of pair bonding. Nat Neurosci 7, 10481054.
  • Young, L.J., Lim, M.M., Gingrich, B. & Insel, T.R. (2001) Cellular mechanisms of social attachment. Horm Behav 40, 133138.