10.1 Estimating Evolutionary Trees

Introduction

The evolutionary history of a group of species is called its phylogeny, and a phylogenetic tree is a graphical summary of this history. The tree describes the pattern and in some cases the timing of events that occurred as the group radiated into new species. The tree also documents which organisms are more closely related, and is an important tool for naming species and genera. Although we sometimes have a fossil record to examine certain morphological traits in some animal groups, this record is fragmentary, and we cannot obtain DNA sequence data except for very recently extinct and well-preserved specimens. We therefore must estimate the phylogenetic tree using extant species. This online unit first reviews the basic logic and methods for estimating trees. It then gives the justification and protocols for the two main uses of trees in animal communication studies: a) correcting for phylogenetic inflation of samples when correlating communication systems with ecological contexts and selection pressures; and b) mapping behavioral or communication signal traits on the phylogenetic tree to infer the sequence of trait evolution. This online unit draws extensively from Chapter 4 of Freeman and Herron’s textbook on evolutionary analysis (Freeman and Herron 2007) and from Hall’s useful manual (2004), and the interested reader should consult these texts for further details.

The logic of phylogenetic inference

The basic logic of inferring evolutionary relationships is that more closely related taxa should have more traits in common. We group species by their similarities and distinguish groups by their differences. In principle, all traits that have a genetic basis and that vary among taxa could be used to assess similarities and differences. Traditionally evolutionary biologists used morphological traits such as bone structures, bristles, and mouthparts, and mode of embryonic or larval development. In the last 25 years, genetic traits such as the presence of certain genes or alleles, the sequence of nucleotides in a particular gene, or the sequence of nucleic acids in a particular protein, have been used with increasing effectiveness. But some traits, loci, DNA/RNA sources, and proteins, are more informative than others.

HOMOLOGY

Homologous characters are those that are similar due to descent from a common ancestor. Phylogenetic reconstruction must of course be based only on homologous traits. The subset of homologies that are most useful for estimating phylogenies are called synapomorphies, which are similar because they are modified from a common ancestor and different from the next more closely related taxon. Synapomorphies help us identify monophyletic groups, also called clades or lineages (Figure 1). Synapomorphies are also useful traits because they identify evolutionary branching points. When two populations of a species become isolated geographically and start to evolve independently, some of the homologous traits in each population will start to diverge due to mutation, selection, and drift. These changed traits are synomorphies that distinguish the two independent lineages. Furthermore, synapomorphies are nested. Over time, multiple branches occur. Each branching event adds one or more shared, derived traits, and the result is a hierarchy of synapomorphies. The clustering of synapomorphies graphed in this way is called a cladogram. A cladogram shows the pattern, or ordered sequence, of cumulative evolutionary change.

Figure 1: A phylogenetic tree with five species. The black lines are branches, which intersect and terminate at nodes. The nodes at the tip represent the five terminal extant species from which we have measured some character traits. The internal nodes (where two branches intersect) represent ancestral species whose traits we can only infer from existing taxa. Red circles enclose each clade or lineage, a monophyletic group of an ancestor plus its descendants. Synapomorphies are nested in a hierarchy to define different taxonomic levels (red arrows). Different synapomorphies define each clade. By convention, the specific character of a synapomorphy and the point where it arose in the evolutionary sequence is indicated on cladograms by a bar across the branches (yellow) and described by accompanying labels or keys. The different synapomorphies also identify the branching points of speciation events (blue arrows).

PROBLEMS IN RECONSTRUCTING PHYLOGENIES

To accurately reconstruct a tree, researchers must identify characters that qualify as synapomorphies. This is not always easy to do, since traits that appear to be similar in two species or taxa may have evolved independently. In this case, the traits were not derived from a common ancestor, so they are neither homologous nor true synapomorphies. One reason for these superficial similarities is convergent evolution, which occurs when natural selection favors similar structures as solutions to an environmental factor. For example, both octopuses and ray-finned fishes possess a camera lens eye, because both depend on excellent vision to find food and detect predators. But the eye structure has evolved independently in the mollusk and the early vertebrate. Convergences can also occur for genetic traits, especially for DNA sequences, where there are only 4 possible nucleotide bases (A, G, T, C) and by chance they can be the same in short sequences for species that are completely unrelated. Mistaking a similar trait for a synapomorphy when it is in fact a convergence constitutes a serious problem for figuring out which species are most closely related.

A second problem is reversal, where a trait that has evolved in a lineage is subsequently lost in a terminal species. Reversals are also common in DNA data, for the same reason mentioned above concerning the likelihood of a mutation back to an earlier nucleotide base given that there are only 4 possibilities. Convergence and reversal are lumped under the term homoplasy. If similar traits are not due to homology then they are due to homoplasy. Homoplasy represents noise in the datasets used to reconstruct phylogenies.

The most effective way to distinguish homology from homoplasy is to use many different traits, or very long sequences, in reconstructing evolutionary relationships. For example, ray-finned fish and other vertebrates have a bony skeleton and a variety of other traits that distinguish them from octopuses and other mollusks. One would have to assume that all of these traits had also changed in concert with the eye to draw the conclusion that fish and mollusks were closely related. It requires many fewer changes to propose that camera lens eyes evolved twice in a convergent process. This type of argument illustrates the principle of parsimony. Parsimony is a general logical criterion whereby simpler explanations for a phenomenon are preferred over more complex ones. When applying parsimony to phylogenetic inference, we accept the tree that involves the fewest changes. While parsimony is logically appealing, sometimes it does not work. For example, trait loss (reversal to an ancestral condition of no trait) could be relatively common for extravagant sexually selected traits that evolved in an ancestor and proved to be very costly to bearers in some populations or species. Researchers therefore need to be very careful in selecting traits for phylogenetic reconstruction that are least subject to homoplasy and more reliable as sources of synapomorphies.

ROOTING THE TREE

A tree is said to be rooted if there is a particular node, the root, from which all other nodes can be reached by moving forward. The root is the hypothetical common ancestor of all the taxa included in the analysis. An unrooted tree specifies only the relationships among the taxa, but it cannot tell us anything about the direction of evolution and the ordering of trait changes over time. Unrooted trees are illustrated with a radial format (Figure 2) because the ancestral node has not been defined. To root a tree, an outgroup species must be included among the set of species in the ingroup, the taxon of interest. An outgroup is a taxon that is more distantly related to each of the ingroup taxa than any of the ingroup taxa are to each other. Selection of a good outgroup taxon is not always easy. It must be sufficiently distantly related to the taxa being considered, but not so distant that it doesn’t share a common ancestor with the ingroup species, i.e., it is not homologous. In practice, researchers typically rely on external sources of information from (earlier) classical studies to identify one or more candidate outgroup species.

Figure 2: An unrooted tree of immunodeficiency viruses in different host species. This tree was created using genomic sequences and a neighbor-joining algorithm. Support at each internal node was assessed using 1000 bootstrap samplings. Branch lengths are based on the number of genetic changes. FIV = feline immunodeficiency virus (blue), SIM = simian immunodeficiency virus (shades of gray), and HIV = human immunodeficiency virus (shades of red). (After Yamamoto et al. 2006.)

DIFFERENT WAYS TO PRESENT TREES

Trees can be presented in different ways, and each has its advantages and disadvantages. A cladogram, which shows only the branching order of nodes, can be drawn in either slanted or rectangular style. Figure 3 illustrates this difference. Either can be used to show the tree topology, and the points at which key adaptations arise can be denoted in a similar way. If the number of species being examined is large, the rectangular style will be easier to display in a compact amount of space.

Figure 3: Slanted and rectangular style of trees. The same taxa and data are shown here using the two techniques: (A) slanted, and (B) rectangular. Both may be oriented horizontally or vertically. A polytomy is included in this example (with species D, E, and F) to show how unresolved nodes, or a multifurcation (as opposed to a bifurcation) is drawn in each case.

When illustrating some type of distance or time information is important, the rectangular style must be used. Such a graph is called a phylogram. When genetic sequence data has been used to create the tree, the branch lengths can be varied to show the number of sequence changes occurring on each branch. The number of changes is considered to be an indication of either evolutionary time or strength of selection (Figure 4a). If something is known about the rate of genetic change, then the x axis can be expressed as time. For example, a molecular clock can be estimated using mutation rates at a selectively neutral gene locus and calibrated to real time if some independent fossil evidence is available. This results in a phylogram where the node positions can indicate the time of divergence between two branches, or the time of speciation events (Figure 4b).

Figure 4: Phylograms with distance/time information. (A) A molecular phylogeny of Zenadia doves, using two other genera of doves as outgroups. Branch lengths reflect genetic distance—shorter branches indicate species or populations that had fewer changes and are considered more closely related (Johnson et al. 2001). (B) Estimated divergence times for the apes, based on a combination of data from dozens of proteins used as molecular clocks. The heavy bars show ±1 standard error around the time estimates and lighter bars show 95% confidence intervals (Stauffer et al. 2001).

FINDING THE BEST TREE

When many traits are used in phylogenetic reconstruction, and the number of species being compared is moderate to large, computer software must be used to construct the tree. PAUP, PHYLIP, MrBayes, BEAST, RAXML, and PHYLLAB are six commonly used packages that build trees using several of the methods mentioned below, and others are more specific to certain methods (for details see Hall 2004, and web resources at end of this unit). The four main methods are neighbor joining, maximum parsimony, maximum likelihood and Bayesian Markov chain Monte Carlo methods. Neighbor joining is an algorithmic method that computes a single best tree, whereas the others are tree-searching methods. In the latter case, many possible trees are identified, and then one must decide which tree is best. Alternative methods involve computing a probability of likelihood that alternative trees are supposed by the data.

Neighbor joining: This is a distance method that converts discrete character data, such as the presence or absence of a morphological trait or the identity of a nucleotide at a homologous location in a gene, into a distance value (or in other words, the inverse of a similarity value). For example, two species are separated by a genetic distance of 10% if an average of 10 nucleotides have change per 100 bases. Many discrete characters can be combined into an overall average distance value. A matrix of pairwise distance estimates is computed between all pairs of taxa, and a clustering algorithm is used to group the most similar taxa (smallest genetic distance) together. A single tree is constructed.

Maximum parsimony: Parsimony can be employed with multiple traits, by summing all of the changes across all of the characters and selecting the tree with the lowest sum, but this strategy may not provide the most reasonable answer. This method typically finds several to many trees with the same, or only slightly different, numbers of changes. With several more or less equally parsimonious trees, one must then use other methods to select among them, or combine them.

Maximum likelihood: The essence of the maximum likelihood method with genetic sequence data is to determine how likely one would obtain a particular set of DNA sequences, given a mathematical formula that describes the probability that different types of nucleotide substitution will occur, and given a particular phylogenetic tree with known branch lengths. A computer can evaluate this question for all possible tree patterns, and the one with the highest likelihood is chosen. RAXML is the best software program to use for this method.

Bayesian analysis: Bayesian approaches are similar to maximum likelihood, except one asks what the probability is of a particular tree being correct, given the data and a model of how the traits in question change over time. Bayesian methods usually produce a set of trees of approximately equal likelihoods. The results are easy to interpret because the frequency of a given clade in that set of trees is identical to the probability of that clade, so no bootstrapping (see below) is required to assess the confidence in the structure of the tree. BEAST is the best software program to use for this method.

Which method is best? Although it seems very efficient to use an algorithmic method that finds the single best tree, one can be misled by not considering other possible trees. The “correct” tree doesn’t exist because we are trying to deduce the order in which existing taxa diverged from a hypothetical common ancestor and the amount of change along the branches between the diverging events. We can’t be sure that a tree topology accurately reflects the historical branching order. The tree-comparing methods are becoming increasingly more popular because they provide a statistical test for estimating how much better one tree is over another. If several very good trees have been obtained with slight differences, one can combine them into a single consensus tree.

Having obtained what we believe is the best tree, we can then evaluate how well supported a particular branch of the tree is using a technique called bootstrapping. This question is analogous to asking about the reliability of measuring height in a group of people. After measuring some people and computing the average of the sample, one evaluates whether the sample average is an accurate representation of the true average for the entire population by computing a measure of the variance around the mean. If the variance is high, one becomes less confident that the sample average is reliable. In bootstrap analysis with trees, one compares trees with and without a particular branch. A computer creates new datasets from the existing one by repeated sampling. With many repeats, one can then compute how many times a particular branch was estimated from the resampled data. If this number is very high, 90–100%, one can be very confident that the branch is accurate. If the number is around 50%, then it is best to conclude that support for the branch is low, and collapse that portion of the tree into a polytomy, or a point of uncertainty, in the published tree (see Figure 3).

Using phylogenies to answer questions

Once a well-supported phylogeny has been obtained for a taxon of interest, the phylogeny can be used to answer a variety of questions. These questions fall into two broad categories: those in which the phylogeny is used to control for taxonomic relatedness so that relationships among traits and ecological variables can be analyzed across species, and those in which the phylogeny is used to generate and test evolutionary hypotheses. We take up these two categories in turn.

COMPARATIVE METHOD STUDIES WITH PHYLOGENETIC CORRECTION

In studies using the comparative method, the investigator compares the quantitative value of two (or more) traits across a range of species, or the association of a trait with some ecological variable, looking for significant correlations. The study may address whether ambient selective pressures play an important role in observed patterns of trait diversity, or whether the relationship with ecological factors meets some theoretical expectation. A bit of history may be useful here.

The field of animal behavior in the 1950s was large focused on relating the diversity of animal behaviors and signals to their phylogeny at the ultimate level and to their inherited physiological mechanisms at the proximate level. Behavioral traits were even proposed as useful criteria for determining or at least confirming, phylogenetic trees. The assumption that behavior was as tightly linked to phylogeny as morphology was shattered by a number of pioneering comparative studies in the 1960s. These included correlations between avian mating systems and habitats by Crook (1964), Brown (1964), Verner and Willson (1966), and Lack (1968); similar correlative studies were published on primates (Crook and Gartlan 1966; Crook and Aldrich-Blake 1968) and antelopes (Jarman 1974). Statistical tests were undertaken to identify which correlations were significant and which not. The resulting significant correlations found among so many taxa triggered efforts to derive evolutionary models that could explain the correlations: Orians' (1969) model for the evolution of polygyny is one important example. Overviews by Emlen and Oring (1977) and Clutton-Brock and Harvey (1977) then sought to integrate process and correlations into general theories of social and mating system evolution in animals. By the late 1970s, it was largely accepted that ecology was usually a better predictor of social, mating, and communication systems than was phylogeny. However, concern arose when Felsenstein (1985) pointed out that such comparative studies among a set of species contained an inherent statistical problem: because species are part of a hierarchically structured phylogeny and related to each other to varying degrees, they are not truly statistically independent samples for correlational studies. Even if the phylogenetic effects are subtle, drawing conclusions from correlations between behavior and habitat for 50 rodents, one ungulate, two primates, and a whale might be a risky business. At best, phylogenetic relatedness adds noise to such comparisons, and at worst, sample inflation by one group of related species might lead to spurious conclusions. Clearly, the comparative method required careful corrections for phylogenetic relationships among those species sampled.

Several techniques have been proposed for how best to go about doing this (Harvey and Pagel 1991; Harvey and Purvis 1991; Garland et al. 1992; Martins and Hansen 1997; Butler and King 2004; Garland et al. 2005) but they boil down to three basic methods: independent contrasts, Monte Carlo computer simulations, and generalized least squares models. Independent contrasts, proposed by Felsenstein, involves taking the difference between the trait values for two adjacent species or groups of species on the tree; each difference is independent of other differences elsewhere on the tree, and therefore can be used as a statistically independent point for correlational analyses (Felsenstein 1985; Garland et al. 1992). Monte Carlo simulation, by randomly moving the trait values around on a given tree, provides an empirical null distribution and defines a critical 5% acceptance value against which the observed correlation can be compared (Martins and Garland 1991; Garland et al. 1993). Phylogenetic generalized least squares regression (PGLS) is currently the favored method. The investigator develops a model of the rate of evolutionary change, such as the basic random Brownian model or the Ornstein-Ulenbeck model that assumes selective optima, and uses this expectation along with an independently derived phylogeny (with known branch lengths) to figure out how much of the variance and covariance among traits can be expected just from phylogenetic history. The residuals are then used in a regression analysis between the traits of interest. This method allows for the exploration of multivariate models. Both MATLAB and R are good software platforms with a variety of existing routines for exploring alternative models (Lavin et al. 2008; PHYSIG, see web resources at end of this unit).

A good communication-related example of the use of PGLS to examine the possible environmental selective pressures affecting signaling traits is the comparative study of song complexity in mockingbirds and their relatives (Mimidae). Ten quantitative measurements were made on song clips of 29 species and reduced with factor analysis to three orthogonal song characters: heterospecific mimicry, short-term note diversity, and song complexity. From the known geographic ranges of each species, the variance and predictability of rainfall and temperature were obtained; habitat type and the migration behavior of each species were also considered. The environmental and song variables were analyzed with multiple regression models while controlling for phylogenetic relatedness. All three of the song variables, but especially mimicry, were significantly associated with more variable and unpredictable climates. Mimicry was especially strongly associated with facultative migration. Mimicry and short-term diversity had strong phylogenetic components. Possible functional explanations for these relationships included stronger selection in more variable and unpredictable climates leading to more elaborated signals of quality, or to signals of intelligence in the mate attraction context (Botero et al. 2009).

Phylogenetic corrections are now expected of any multispecies comparative study. In a recent review of 194 comparative studies that also reported the raw (uncorrected) result, the phylogenetic correction changed the conclusion in only a few cases (Garamszegi and Møller 2010). Moreover, the direction of the change after correction either improved or diminished the correlations in roughly equal numbers of cases. This review article also points out that most studies neglect the effects of within species variation. Heterogeneous sampling within species can potentially be as important as phylogenetic effects in comparative analyses. Some additional comparative analyses of communication-related traits include the following: Aparicio et al. 2003; Hagman and Forsman 2003; Kraaijeveld 2003; Galeotti et al. 2003; Bleiweiss 2004; Stuart-Fox and Ord 2004; Emlen et al. 2005; Olson and Owens 2005; Seddon 2005; Soler et al. 2005; Richards 2006; Berg et al. 2006; Bokony et al. 2007; Del Castillo and Gwynne 2007; Mank 2007; Doucet et al. 2007.

PHYLOGENY-DRIVEN HYPOTHESES

Several kinds of questions are driven by explicit knowledge of the phylogenetic tree. Evolutionary biologists of course want to know which species form meaningful clades so they can name them correctly. This entails determining whether taxa classified and named using classical morphological characters still hold up once genetically-based trees have been made (de Queiroz and Cantino 2001). Why certain species are found in certain parts of the world and how geographic distributions have changed through time can be examined with phylogenetic analyses (Pagel 1999; Raxworthy et al. 2002). Co-speciation between parasites and their hosts is another interesting question, in which independent trees are built for both groups, and one asks whether they speciated together (Clark et al. 2000). Behavioral biologists would like to know whether and which behavioral traits are useful taxonomic characters (de Queiroz and Wimberger 1993; Slikas 1998; Blomberg et al. 2003). Animal communication researchers often seek the ancestral form of signaling traits and the progression of signal elaboration across species (Phelps and Ryan 2000). Finally, we need very accurate trees to ask whether a signal trait or a receiver trait evolved first.

For questions that involve mapping traits onto a tree and assessing their evolutionary pattern across clades, one needs a quantitative measure of the degree of homology (or homoplasy). Several such measures have been developed. A simple and very useful measure, especially for discrete traits, is the rescaled consistency index, which quantifies the minimum amount of change a character may show on any tree divided by the observed number of changes, scaled by the maximum possible amount of homoplasy (Kluge and Farris 1969; Farris 1989a, b). The index has a value of 0 when the character is completely homoplasious, and 1 when the character is perfectly homologous. Another type of measure is called the test for serial independence (Abouheif 1999). It measures the degree of non-randomness in a sequence of trait values for the terminal taxa in a tree. The variation in the differences between successive taxa is divided by the sum of squares of the observations. Completely random sequences have a value of 2 (i.e., homoplasious), a value less than 2 indicates homology, and a value greater than 2 indicates non-random alternation. This test is independent of branch lengths and any type of model of evolutionary change, which has been considered a problem. Other randomization tests have been devised that allow such refinements. Here, the trait values for the terminal taxa are permuted many times to obtain a random expectation, and observed values are compared. Variants of these models can take into account different branch lengths on the tree, and different evolutionary models for the tempo of evolutionary change, either accelerating or decelerating (Blomberg et al. 2003). Another strategy is to compute a measure of similarity between all pairs of terminal taxa using one or more traits, and then correlate these values with the genetic distances between the same pairs of taxa (Slabbekoorn et al. 1999). This technique has the usual problem with spurious and confounding correlations. For example, a correlation between genetic distance and a display trait character could be a spurious result of a true correlation between the display trait and habitat features, because habitat is correlated with genetic distance (Rheindt et al. 2004). Regardless of which method is used, when there is a significant amount of homology for a trait mapped onto a tree, we conclude that there is a strong phylogenetic signal (not to be confused with a communication signal).

Below are a four interesting examples of studies in which communication signals have been mapped onto trees to evaluate the pattern of evolution of the traits. The first example (Figure 5) examined song characteristics of oropendolas, neotropical songbirds with black and yellow plumage in the Icterid family. The well-supported tree was derived from DNA sequence data, and branch lengths reflect molecular changes. Genera include Psarocolius, Gymnostinops, and Ocyalus. Of 32 acoustic features examined, the 22 shown here revealed unambiguous evolutionary changes (i.e., significant homology). Many of these traits evolved once and were subsequently retained in the clade, The two lower taxa have complex song repertoires, while the remaining species all have one song type (11) and perform the bow display (9). The trill (6) evolved once and now defines a subclade comprising P. atrovirens and the three P. angustifrons subspecies. The harsh broadband crash note (7) showed a reversal, arising in the upper clade and then was lost in two of the P. a. atrovirens subspecies. Multiple evolutionary changes occurred in the branch for G. montezuma, a highly polygynous species whose song is long and intense with large frequency shifts and note overlap. The variable characters based on frequency properties of songs (26, 28–31) were uniquely derived in terminal taxa; this implies rapid evolutionary change of these features that are the most vulnerable to degradation during transmission in different habitats.

(1) Introductory rattle	(11) Song versatility	(23) Rattle rate
(3) Broadband rattle	(12) Intersong interval	(24) Trill note rate
(4) Rattle-whistle	(15) Song duration	(25) Highest peak frequency
(5) Click	(16) Note percentage	(28) Frequency range
(6) Trill	(18) Note overlap	(29) Frequency shift rate
(7) Crash	(20–21) Pause duration	(30) Max frequency shift
(9) Bow	(22) Pause rate	(31) Frequency slope

Figure 5: Phylogenetic analysis of the evolution of song features in oropendolas. Song traits shown enumerated in the table are mapped onto the phylogeny, with green indicating traits that were gained and red indicating traits that were lost. Spectrograms of male songs are shown on the right. (After Price and Lanyon 2002.)

Another interesting phylogenetic analysis examined the acoustic structure of the male call in two related groups of Australian psyllids (Hemiptera: Psylloidae, family Triozidae), plant-sucking insects related to whiteflies and aphids (Figure 6). One group, contained in the genus Schedotrioza, feeds on Eucalyptus; species are allopatric by microhabitat segregation because each feeds on a different host plant species. The other group is more heterogeneous morphologically and feeds on various host plants in the family Casuarinaceae; species in this group all feed on the same set of host plant species, often on the same individual plant, and are therefore usually sympatric with each other. The sounds are produced by stridulation using forewing vibrations, which are primarily transmitted through the plant substrate. In many species, females answer male calls with their own unique stridulatory sounds, and courting pairs engage in antiphonal duetting. The phylogeny shows the Schedotrioza to be a monophyletic and more recently derived genus, with a distinctive male call structure defining the clade. Furthermore, the correlation between genetic distance and acoustic distance was positive within the group, as it was when all species were combined, indicating a strong phylogenetic signal in the call structure. However, within the Casuarinaceae-feeding species, the genetic–acoustic distance correlation was negative, which may reflect increased selection for acoustic divergence in closely related, sympatric taxa. The phylogenetic tree indicates that this is probably not a monophyletic group (Percy et al. 2006).

Figure 6: Phylogenetic analysis of acoustic stridulatory signals in Australian psyllids. (A) Single tree obtained from maximum parsimony analysis of mitochondrial DNA sequences. Host plant groups are indicated by color-coded branches of the psyllid tree and host names are given after the psyllid name. Male call shown on right. (B) Acoustic distance between all pairs of the 11 species studied acoustically is positively correlated with genetic distance for all species combined (black line). Within Schedotrioza (red) the correlation is positive, but is negative within the Casuarinaceae-feeding group (blue). Purple circles denote comparisons between pairs of species from different groups. (From Percy et al. 2006.)

The third study examined plumage color patterns in the New World orioles (genus Icterus). Each species exhibits a unique pattern of black, orange or yellow, and white color patches, yet they vary within similar themes. There are two common themes found in several sets of species, the Baltimore type with black heads and white edging on most wing coverts, and the Altamira type with carotenoid-colored heads and white patches on the outer primaries. This study attempted to determine whether these plumage types reflected common ancestry (synapomorphy) or independently derived convergence or reversal (homoplasy). The molecular phylogeny was based on DNA sequence data from two mitochondrial genes, and revealed three clades within the genus. Forty-four feather patch areas were scored as being white, black, or carotenoid. Each one was separately mapped onto the phylogeny and various measures of homology (consistency index CI and percent phylogenetic signal strength) were computed for each. Figure 7 shows the mapping of one patch—the coloration of the lesser wing coverts—which had one of the lowest CI values and appeared repeatedly in all three of the main clades, yet there was still a significant phylogenetic signal in this character. The table summarizes these two measures for all 44 patches. Most patch areas showed repeated reversals and convergences. The Baltimore plumage type included species from clades C and A, and the Altamira type included species from all three clades. This finding indicates that distantly related species frequently converge on a similar overall pattern. Despite the generally high level of homoplasy for individual patches, there were few uniquely derived character states. The researchers concluded that there was a surprising amount of plumage lability at one level, but a strong degree of plumage conservatism at another level (Omland and Lanyon 2000).

Figure 7: Plylogeny of color patches among orioles. (A) Tree reconstructed from molecular sequence data, with color states of the lesser wing coverts indicated. (B) Table showing consistency index (CI) and percent phylogenetic signal for all color patches. (After Omland and Lanyon 2000.)

The final example of a phylogenetic approach to signal evolution deals with the waveform of electric pulses in African Mormyroid elephantfishes (Sullivan et al. 2000). The electric discharge is used for both electrolocation and for communicating with conspecifics. The pulses are discharged continuously and have a consistent species-specific shape. In this phylogenetic example, the morphology of the electric signal-producing organ has also been determined for each species and the innervation pattern is associated with the waveform shape, so the precise evolutionary pattern of changes in the signal generation mechanism can also be deduced. The tree shown in Figure 8 comprises 41 species found in west-central Africa. The sister group to the Mormyrid elephantfishes is the Gymnarchids, which has a primitive electric organ.

The electric organ in this taxon consists of disk-like cells called electrocytes arranged in a series of columns in the narrow region of the tail. The flat surface of the disks is oriented perpendicular to the skin surface. A nerve stalk innervates each disk, and disks in a column are simultaneously stimulated. Electrocytes can be classified into six types based on the presence or absence of penetrating stalks and the side of innervation. The type S cell is stalkless (innervated by a simple branching nerve as shown in the figure) and produces a monophasic pulse. This type is found in the Gymnarchus outgroup. There are two major categories of stalked electrocytes. In the first, stalks arising from the posterior face of the electrocyte disk are non-penetrating and receive innervation on the posterior side (type NPp). All fish with this type produce simple biphasic electric organ discharge waveforms. In the second category, the stalk penetrates through the electrocyte. In the simplest of these, the stalk penetrates through the electrocyte to the opposite side, so that it receives innervation from the anterior side (type Pa). Others have stalks arising from the anterior face that penetrate through to receive innervation from the posterior face (type Pp). These two types produce biphasic pulses. In Brienomyrus, the stalk penetrates through the electrocyte a second time to receive innervation on the face from which it originated (called a doubly penetrating stalk with posterior innervation, type DPp). This arrangement produces a triphasic pulse. Species in one genus, Pollimyrus, have both doubly penetrating and non-penetrating stalks (type DPNP). The figure shows the point at which each of these electrocyte innervation strategies is believed to have arisen. What is especially interesting in this phylogenetic analysis is that is it very easy for reversals to occur, i.e., for a multiple penetrating stalk to revert to a simpler form with fewer penetrations, because these represent more primitive developmental stages.

Figure 8: Phylogeny of electric organ evolution in Mormyroid elephantfishes. The robust phylogenic tree is based on genetic sequences from 4 gene loci. The six electric organ morphological types and typical waveform EOD that they might generate are illustrated on the left, and arrows point to the hypothesized position in the tree where they first arose. Numbers refer to hypothesized character state changes. Note the frequent occurrence of reversals, from the typical Pa organ type shown in red to the simpler, ancestral NPp type shown in black. (From Sullivan et al. 2000. Reproduced with permission from the Journal of Experimental Biology. Colored figure courtesy of Carl Hopkins.)

Web resources

PAUP: http://paup.sc.fsu.edu/

PHYLIP: http://www.phylip.com/

MrBayes: http://mrbayes.csit.fsu.edu/

BEAST: http://beast.bio.ed.ac.uk/

RAXML: http://www.embl.de/~seqanal/courses/commonCourseContent/usingRaxml.html and
http://phylobench.vital-it.ch/raxml-bb/

R packages: http://www.r-phylo.org/wiki/Main_Page

PHYSIG: http://www.biology.ucr.edu/people/faculty/Garland/PHYSIG.html

MESQUITE: https://mesquiteproject.wikispaces.com/

Literature cited

Abouheif, E. 1999. A method for testing the assumption of phylogenetic independence in comparative data. Evolutionary Ecology Research 1: 895–909.

Aparicio, J.M., R. Bonal and P.J. Cordero. 2003. Evolution of the structure of tail feathers: Implications for the theory of sexual selection. Evolution 57: 397–405.

Berg, K.S., R.T. Brumfield and V. Apanius. 2006. Phylogenetic and ecological determinants of the neotropical dawn chorus. Proceedings of the Royal Society B-Biological Sciences 273: 999–1005.

Bleiweiss, R. 2004. Novel chromatic and structural biomarkers of diet in carotenoid-bearing plumage. Proceedings of the Royal Society B-Biological Sciences 271: 2327–2335.

Blomberg, S.P., T. Garland and A.R. Ives. 2003. Testing for phylogenetic signal in comparative data: Behavioral traits are more labile. Evolution 57: 717–745.

Bokony, V., A. Liker, T. Szekely and J. Kis. 2003. Melanin-based plumage coloration and flight displays in plovers and allies. Proceedings of the Royal Society of London Series B-Biological Sciences 270: 2491–2497.

Botero, C.A., N.J. Boogert, S.L. Vehrencamp and I.J. Lovette. 2009. Climatic patterns predict the elaboration of song displays in mockingbirds. Current Biology 19: 1151–1155.

Bro-Jørgensen, J. 2007. The intensity of sexual selection predicts weapon size in male bovids. Evolution 61: 1316–1326.

Brown, J.L. 1964. The evolution of diversity in avian territorial systems. The Wilson Bulletin 76:160–169.

Butler, M.A. and A.A. King. 2004. Phylogenetic comparative analysis: A modeling approach for adaptive evolution. American Naturalist 164: 683–695.

Clark, M.A., N.A. Moran, P. Baumann and J.J. Wernegreen. 2000. Cospeciation between bacterial endosymbionts (Buchnera) and a recent radiation of aphids (Uroleucon) and pitfalls of testing for phylogenetic congruence. Evolution 54: 517–525.

Clutton-Brock, T.H. and P.H. Harvey. 1977. Primate ecology and social organization. Journal of Zoology (London) 183:1–39.

Crook, J.H. 1964. The evolution of social organization and visual communication in the weaver birds (Ploceinae). Behaviour Suppl. 10: 1–178.

Crook, J.H. and P. Aldrich-Blake. 1968. Ecological and behavioral contrasts between sympatric ground dwelling primates in Ethopia. Folia Primatologica 8: 192–227.

Crook, J.H. and J.S. Gartlan. 1966. Evolution of primate societies. Nature 210: 1200–1203.

de Queiroz, A. and P.H. Wimberger. 1993. The usefulness of behavior for phylogeny estimation — Levels of homoplasy in behavioral and morphological characters. Evolution 47: 46–60.

de Queiroz, K. and P.D. Cantino. 2001. Phylogenetic nomenclature and the Phylocode. Bulletin of Zoological Nomenclature 58: 254–271.

Del Castillo, R.C. and D.T. Gwynne. 2007. Increase in song frequency decreases spermatophore size: correlative evidence of a macroevolutionary trade-off in katydids (Orthoptera: Tettigoniidae). Journal of Evolutionary Biology 20: 1028–1036.

Doucet, S.M., D.J. Mennill and G.E. Hill. 2007. The evolution of signal design in manakin plumage ornaments. American Naturalist 169: S62–S80.

Emlen, D.J., J. Marangelo, B. Ball and C.W. Cunningham. 2005. Diversity in the weapons of sexual selection: Horn evolution in the beetle genus Onthophagus (Coleoptera: Scarabaeidae). Evolution 59: 1060–1084.

Emlen, S.T. and L.W. Oring. 1977. Ecology, sexual selection, and the evolution of mating systems. Science 197:215–223.

Farris, J.S. 1989a. The retention index and homoplasy excess. Systematic Zoology 18: 374–385.

Farris, J.S. 1989b. The retention index and the rescaled consistency index. Cladistics-the International Journal of the Willi Hennig Society 5: 417–419.

Felsenstein, J. 1985. Phylogenies and the comparative method. American Naturalist 125: 1–15.

Freeman, S. and J.C. Herron. 2007. Evolutionary Analysis. Upper Saddle River, NJ: Pearson Prentice Hall.

Galeotti, P., D. Rubolini, P.O. Dunn and M. Fasola. 2003. Colour polymorphism in birds: causes and functions. Journal of Evolutionary Biology 16: 635–646.

Garamszegi, L.Z. and A.P. Møller. 2010. Effects of sample size and intraspecific variation in phylogenetic comparative studies: a meta-analytic review. Biological Reviews 85: 797–805.

Garland, T., A.F. Bennett and E.L. Rezende. 2005. Phylogenetic approaches in comparative physiology. Journal of Experimental Biology 208: 3015–3035.

Garland, T., A.W. Dickerman, C.M. Janis and J.A. Jones. 1993. Phylogenetic analysis of covariance by computer simulation. Systematic Biology 42: 265–292.

Garland, T., P.H. Harvey and A.R. Ives. 1992. Procedures for the analysis of comparative data using phylogenetically independent contrasts. Systematic Biology 41: 18–32.

Hagman, M. and A. Forsman. 2003. Correlated evolution of conspicuous coloration and body size in poison frogs (Dendrobatidae). Evolution 57: 2904–2910.

Hall, B.G. 2004. Phylogenetic Trees Made Easy. Sunderland, MA: Sinauer Associates, Inc.

Harvey, P.H. and M.D. Pagel. 1991. The Comparative Method in Evolutionary Biology. Oxford: Oxford University Press.

Harvey, P.H. and A. Purvis. 1991. Comparative methods for explaining adaptations. Nature 351: 619–624.

Jarman, P.J. 1974. The social organization of antelope in relation to their ecology. Behaviour 48:215–267.

Johnson, K.P., S. De Kort, K. Dinwoodey, A.C. Mateman, C. Ten Cate, C.M. Lessells and D.H. Clayton. 2001. A molecular phylogeny of the dove genera Streptopella and Columba. Auk 118: 874–887.

Kluge, A.G. and J.S. Farris. 1969. Quantitative phyletics and the evolution of anurans. Systematic Zoology 18: 1–32.

Kraaijeveld, K. 2003. Degree of mutual ornamentation in birds is related to divorce rate. Proceedings of the Royal Society B-Biological Sciences 270: 1785–1791.

Lack, D. 1968. Ecological Adaptations for Breeding in Birds. Methuen: London UK.

Lavin, S.R., W.H. Karasov, A.R. Ives, K.M. Middleton and T. Garland. 2008. Morphometrics of the avian small intestine compared with that of nonflying mammals: A phylogenetic approach. Physiological and Biochemical Zoology 81: 526–550.

Mank, J.E. 2007. The evolution of sexually selected traits and antagonistic androgen expression in actinopterygiian fishes. American Naturalist 169: 142–149.

Martins, E.P. and T. Garland. 1991. Phylogenetic analyses of the correlated evolution of continuous characters — a simulation study. Evolution 45: 534–557.

Martins, E.P. and T.F. Hansen. 1997. Phylogenies and the comparative method: A general approach to incorporating phylogenetic information into the analysis of interspecific data. American Naturalist 149: 646–667.

Olson, V.A. and I.P.F. Owens. 2005. Interspecific variation in the use of carotenoid-based coloration in birds: diet, life history and phylogeny. Journal of Evolutionary Biology 18: 1534–1546.

Omland, K.E. and S.M. Lanyon. 2000. Reconstructing plumage evolution in orioles (Icterus): Repeated convergence and reversal in patterns. Evolution 54: 2119–2133.

Orians, G.H. 1969. On the evolution of mating systems in birds and mammals. The American Naturalist 103:589–603.

Pagel, M. 1999. Inferring the historical patterns of biological evolution. Nature 401: 877–884.

Percy, D.M., G.S. Taylor and M. Kennedy. 2006. Psyllid communication: acoustic diversity, mate recognition and phylogenetic signal. Invertebrate Systematics 20: 431–445.

Phelps, S. M. and M. J. Ryan. 2000. History influences signal recognition: neural network models of túngara frogs. Proceedings of the Royal Society of London Series B-Biological Sciences 267: 1633–1639.

Price, J.J. and S.M. Lanyon. 2002. Reconstructing the evolution of complex bird song in the oropendolas. Evolution 56: 1514–1529.

Raxworthy, C.J., M.R.J. Forstner and R.A. Nussbaum. 2002. Chameleon radiation by oceanic dispersal. Nature 415: 784–787.

Rheindt, F.E., T.U. Grafe and E. Abouheif. 2004. Rapidly evolving traits and the comparative method: how important is testing for phylogenetic signal? Evolutionary Ecology Research 6: 377–396.

Richards, C.L. 2006. Has the evolution of complexity in the amphibian papilla influenced anuran speciation rates? Journal of Evolutionary Biology 19: 1222–1230.

Seddon, N. 2005. Ecological adaptation and species recognition drives vocal evolution in neotropical suboscine birds. Evolution 59: 200–215.

Slabbekoorn, H., S. De Kort and C. Ten Cate. 1999. Comparative analysis of perch-coo vocalizations in Streptopelia doves. Auk 116: 737–748.

Slikas, B. 1998. Recognizing and testing homology of courtship displays in storks (Aves: Ciconiiformes: Ciconiidae). Evolution 52: 884–893.

Soler, J.J., J. Moreno, J.M. Aviles and A.P. Moller. 2005. Blue and green egg-color intensity is associated with parental effort and mating system in passerines: Support for the sexual selection hypothesis. Evolution 59: 636–644.

Stauffer, R.L., A. Walker, O.A. Ryder, M. Lyons-Weiler and S.B. Hedges. 2001. Human and ape molecular clocks and constraints on paleontological hypotheses. Journal of Heredity 92: 469–474.

Stuart-Fox, D.M. and T.J. Ord. 2004. Sexual selection, natural selection and the evolution of dimorphic coloration and ornamentation in agamid lizards. Proceedings of the Royal Society of London Series B-Biological Sciences 271: 2249–2255.

Sullivan, J.P., S. Lavoue and C.D. Hopkins. 2000. Molecular systematics of the African electric fishes (Mormyroidea: Teleostei) and a model for the evolution of their electric organs. Journal of Experimental Biology 203: 665–683.

Verner, J. and M.F. Willson. 1966. The influence of habitats on mating systems of North American passerine birds. Ecology 47:143–147.

Yamamoto, J.K., E. Sato, R. Pu, J. Coleman and M. Martin. 2006. Feline immunodeficiency virus vaccines: Evolving concepts in vaccine approaches. Cellscience Reviews 2: 35–49.

10.2 Neural Network Models and Feature Detectors

Introduction

Sensory systems evolve to enable organisms (plant and animal) to gather meaningful information from their environment and then integrate that information to make adaptive decisions. The field of sensory ecology studies how sensory systems are adapted to detect the objects important to a given species’ survival and reproduction, such as food, mates, predators, and shelters. All modalities can be studied from this perspective, and a typical goal is to describe how sensory receptors and processing centers are designed to maximize performance on a given task. For example, the peak pigment sensitivities of marine fishes are optimally designed to maximize luminance and chromatic detection in different habitats (Cummings 2004), the basilar papilla of frogs’ ears is tuned to be most sensitive to the dominant frequency of conspecific male calls (Capranica et al. 1973; Capranica and Moffat 1983; Wilczynski et al. 1992), certain chemical sensors on male moth antennae are exquisitely tuned to detect the pheromone blend of conspecific females, and touch receptors may be highly developed to rapidly detect prey (Catania and Kaas 1997; Catania and Remple 2005).

Feature Detectors

Very specific detection systems are often called feature detectors. Feature detectors are refinements in the receiving apparatus that make the detection of cues and signals easier against background noise. Such receiver detection systems are more than a peripheral filter; they typically involve higher levels of neuronal integration as well (Martin 1994; Ison and Quiroga 2008). Lateral inhibition between adjacent receptor cells plays a key role in feature detector wiring. For example, the retina employs horizontal and amacrine cells to enhance the detection of edges, horizontal or vertical bars, spots, and certain types of movement (Yang and Wu 1991; Cook and McReynolds 1998; Zhang and Wu 2009; Lipin et al. 2010). Similar systems have been described for auditory and olfactory receivers. Owls have feature detectors for locating the position of prey by the sounds they make (Konishi 1993), and species-specific templates for the songs of passerine birds ensure that only conspecific notes and syntax patterns are learned (Lewicki and Konishi 1995; Huetz et al. 2004; Theunissen et al. 2004; Lehongre and Del Negro 2011). The vertebrate olfactory system also uses lateral inhibition to enhance the recognition of certain odorants (Lledo et al. 2004).

Feature detectors are hard-wired in the sense that no learning is involved, although the neuronal pathways may need to be activated with general sensory stimulation. Specific responses to the detection of a feature detector are also often hard-wired. Hard-wiring ensures that the signal or cue is recognized the first time it is encountered and that the appropriate response (approach or evasion) is made. Feature detectors can evolve as part of the receiver tuning process of signal evolution described in main text Figure 10.2 to enhance the detection of important social signals. Here they serve in the recognition of sign stimuli, or releasers for appropriate responsive behaviors. A classic example of a sign stimulus and its corresponding feature detector is the red spot on the bill of many gulls that releases begging behavior by chicks (Tinbergen and Perdeck 1950). Feature detectors can also evolve to facilitate approach toward cues from prey or avoidance of cues from enemies. Once such feature detectors for environmental cues have evolved, they can be exploited by senders that match the key cue characteristics via the receiver-precursor scenario of signal evolution outlined in main text Figure 10.16.

Neural Network Models

Artificial neural network models can be used to explore some of the properties of sensory recognition systems in animals. Neural network models attempt to mimic the basic features of sensory systems and have been used to understand both the key mechanisms underlying sensory recognition and the hidden preferences that may emerge as a byproduct of these mechanisms (Gurney 2007; Phelps 2007). An example of a simple feed-forward model is shown in Figure 1. It consists of three layers. The input layer is a 6 x 6 array of sensory receptor cells analogous to a retina (or olfactory epithelium or basilar papilla). The second layer is called the hidden layer, in this case comprised of 36 cells analogous to ganglion cells. Each cell in this layer is connected to every cell in the input layer above it and each input layer cell is connected to every hidden layer cell. These connections are associated with quantitative weightings that regulate the strength of the signal passing between cells, analogous to a synaptic nerve connection. The total stimulus to a hidden layer cell is therefore the weighted sum of the input from all of the sensory cells in the layer above. The third layer consists of a single output cell connected to all of the hidden layer cells. Input to this output cell is also a weighted sum of the connection strengths from the hidden layer.

Figure 1: A feedforward neural network model for visual mate detection. In the input layer, the blue structure represents the generic characteristics of a conspecific bird with a long tail. In the hidden layer, each cell receives input from every cell in the input layer, shown by red lines. For simplicity, the connections for only a few cells are shown. The output layer receives input from each cell in the hidden layer (encompassed within the orange lines).

When the input layer is stimulated, each of the receptor cells receives a score of zero or one. The output from these cells is then multiplied by the weighting factors to determine the total input to the hidden layer cells. The summed output from hidden layer cells is similarly multiplied by the weighting factors associated with this layer to determine the summed stimulation of the output cell. The output cell is the decision-maker — if the summed input to this cell is above a certain threshold, it responds with a “yes” and the network is said to “recognize” the stimulus pattern. The weighting factors at both levels “evolve” during training sessions in a process that mimics either learning or natural selection to achieve a certain level of discrimination. The input layer is presented with the target object in various locations and orientations and “taught” that this is the correct image using a back propagation procedure or using a genetic algorithm to assign fitness scores. The input layer is also presented with non-target images, which may be a different pattern or random patterns, and taught to reject such images. Networks with weighting patterns that make more correct decisions have higher fitness. Weights undergo random mutational changes at each trial (generation) and the networks with better discrimination abilities are retained for the next trial. Each iterated run produces a unique successful network with a different final set of weighting factors that work (enable the network to make an accurate discrimination); thus there are many solutions to the same problem.

Novel stimulus patterns are then presented to the trained networks without modifying the weightings. A network model is considered robust if it can generalize, i.e., recognize stimuli that are similar to the training target. In other words, it should be able to evolve internal representations that correspond to generic features of the target. Occasionally a network is found that responds more strongly to an exaggerated version of the training pattern. For example, a classic model by Enquist and Arak (1993), using a network structure much like the one depicted in Figure 1, trained the retina to accept an image of a long-tailed bird (with a two-square tail) and to reject a heterospecific bird with a shorter tail (a one-square tail). When the net was presented with a bird with a very long three-square tail, it responded more strongly than it did to the two-square-tail training target. At the time, this study was touted as evidence of potential sensory biases that could explain the elaboration of sexually selected male traits. Such sensory biases could also provide an explanation for the classical ethologist’s observation of supernormal stimuli (Tinbergen 1948). These are the preferences sometimes shown for cues or signal features that are enlarged or exaggerated beyond the normal range (McClintock and Uetz 1996; Burley and Symanski 1998; Drǎgǎnoiu et al. 2002; Ghirlanda and Enquist 2003). Another emergent property of neural net models is an occasional strong response to a novel pattern. No pattern recognition system performs perfectly since backgrounds vary and the pattern may appear in different orientations or at different distances. However, a variety of neural connection schemes may satisfactorily distinguish the pattern from its background on the majority of occasions. Some schemes by chance may be more sensitive to a previously unencountered pattern. This phenomenon has been termed hidden preferences or latent biases (Enquist and Arak 1993). Other network models found similar biases towards symmetrical patterns (Enquist and Arak 1994; Johnstone 1994). Supernormal stimuli and hidden preferences might drive the evolution of new signal forms as proposed by the sensory exploitation model.

Some criticisms have been leveled at the conclusions drawn from these simple neural network models. First, the results may depend on the details of the model. A re-examination of the Enquist and Arak (1993) model using a somewhat different training method and stronger selection regime found that the response to the extra-long tail was not greater than the response to the training tail (Kamo et al. 1998). Kamo et al. attempted to pinpoint the reason for this discrepancy by exploring effects of different training periods and backgrounds. Their network generalized well to similar images, but responses to exaggerated images with longer tails or wings fluctuated wildly over time. They suggested that Enquist and Arak’s network was insufficiently trained. Second, such simple networks are not designed to perceive particular shapes or spatial patterns, but only to respond to the average level of stimulation to a given sensory cell (Cook 1995; Dawkins and Guilford 1995). A true feature detector that perceives spatial pattern shapes must consist of a more complex hidden layer with a two-dimensional array of cells and a third series of weighting factors among the cells in this layer. Like horizontal and amacrine cells of the retina, these connections produce patterns of reinforcement and inhibition between adjacent cells. Neural network models of this type can evolve edge, spot and bar detectors similar to those described by neurophysiologists (Rubner and Schulten 1990).

Fuller (Fuller 2009) employed a more complex network to investigate the feasibility of the sensory bias model of mating preference evolution, whereby males develop traits that match female sensory biases for food items. The model species for this simulation study was the bluefin killifish Lucania goodei, in which males are polymorphic for blue, red, or yellow anal fins. Potential food items could be blue, red, or yellow worms, and the question addressed was whether selection for a colored worm preference could have pleiotropic effects on male trait evolution, given that both foraging and mating use the same visual sensory system. The network was modeled after a color-sensitive retina with five visual pigments (UV, violet, blue, yellow, and red) as found in the killifish. The input layer consisted of a 12 by 12 array with a mosaic arrangement of 144 cone cells, some of which were linked into double-cone units. The next layer was modeled after bipolar cells that connected to a subset of same-pigment cone cells, as well as some mosaic bipolar cells receiving input from the double-cone units. Another layer contained horizontal cells that made lateral connections among cone cells. The next layer represented ganglion cells that received input from all bipolar cells, and another layer represented processing neurons in the brain. Finally, these processing neurons connected to two output neurons, one for mate recognition and one for food recognition. Each cell in the network also connected to a single bias neuron that weighted its threshold for firing. Mutations could occur on both the weightings and the pigment wavelength peaks. Selection on populations of 100 networks was strong, with half of the most poorly performing networks eliminated in each generation, while the remaining networks were allowed to “breed” in a pairwise fashion with high heritability of the “parental” network values. In one series of simulations, the networks were trained to prefer one color of worms over the other colors, and then they were assayed for their mate choice preferences. A small but significant preference for mates having the same colored fins was obtained, but there was a great deal of variation and many cases of preference for alternate colors. This result provided modest support for the correlated responses predicted by the sensory bias hypothesis. However, in other simulations that selected simultaneously on mate and food preference colors, there was no tendency for matching colors to evolve preferentially, showing that foraging and mate choice preferences are not constrained to evolve in concert. Even though such complex network models are still great simplifications of natural evolutionary processes, this study nevertheless demonstrated the restrictive conditions under which correlated responses could evolve.

There are many other very interesting applications of neural network modeling that address communication-related questions, as well as other types of ecological questions. For example, (Hurd et al. (1995) found support for an antithetical (opposite or polarized) form of aggressive versus submissive threat signals, Enquist and Arak (1994) documented the latent attractiveness of symmetrical patterns, Phelps and Ryan (2000) found strong biases for ancestral acoustic signal forms in frogs, and Brodin and Haas (2006) studied the process of sexual imprinting in sympatric speciation. The interested reader can find many useful articles in a Theme Issue of the March 27, 2007 Philosophical Transactions of the Royal Society B (volume 362, issue 1479).

Literature cited

Brodin, A. and F. Haas. 2006. Speciation by perception. Animal Behaviour 72: 139–146.

Burley, N.T. and R. Symanski. 1998. “A taste for the beautiful”: Latent aesthetic mate preferences for white crests in two species of Australian grassfinches. American Naturalist 152: 792–802.

Capranica, R.R., L.S. Frishkof and E. Nevo. 1973. Encoding of geographic dialects in the auditory system of the cricket frog. Science 182: 1272–1275.

Capranica, R.R. and A.J.M. Moffat. 1983. Neurobehavioral correlates of sound communication in anurans. In Advances in Vertebrate Neuroethology (Ewert, J.P., R.R. Capranica and D.J. Ingle, eds). New York: Plenum Press. pp. 701–730.

Catania, K.C. and J.H. Kaas. 1997. Somatosensory fovea in the star-nosed mole: Behavioral use of the star in relation to innervation patterns and cortical representation. Journal of Comparative Neurology 387: 215–233.

Catania, K.C. and F.E. Remple. 2005. Asymptotic prey profitability drives star-nosed moles to the foraging speed limit. Nature 433: 519–522.

Cook, N.D. 1995. Artifact or network evolution? Nature 374: 313–313.

Cook, P.B. and J.S. McReynolds. 1998. Lateral inhibition in the inner retina is important for spatial tuning of ganglion cells. Nature Neuroscience 1: 714–719.

Cummings, M.E. 2004. Modelling divergence in luminance and chromatic detection performance across measured divergence in surfperch (Embiotocidae) habitats. Vision Research 44: 1127–1145.

Dawkins, M.S. and T. Guilford. 1995. An exaggerated preference for simple neural-network models of signal evolution. Proceedings of the Royal Society of London Series B-Biological Sciences 261: 357–360.

Drǎgǎnoiu, T.I., L. Nagle and M. Kreutzer. 2002. Directional female preference for an exaggerated male trait in canary (Serinus canaria) song. Proceedings of the Royal Society B-Biological Sciences 269: 2525–2531.

Enquist, M. and A. Arak. 1993. Selection of exaggerated male traits by female aesthetic senses. Nature 361: 446–448.

Enquist, M. and A. Arak. 1994. Symmetry, beauty and evolution. Nature 372: 169–172.

Fuller, R.C. 2009. A test of the critical assumption of the sensory bias model for the evolution of female mating preference using neural networks. Evolution 63: 1697–1711.

Ghirlanda, S. and M. Enquist. 2003. A century of generalization. Animal Behaviour 66: 15–36.

Gurney, K. 2007. Neural networks for perceptual processing: from simulation tools to theories. Philosophical Transactions of the Royal Society B-Biological Sciences 362: 339–353.

Huetz, C., C. Del Negro, K. Lehongre, P. Tarroux and J.M. Edeline. 2004. The selectivity of canary HVC neurons for the Bird’s Own Song: Rate coding, temporal coding, or both? Journal of Physiology–Paris 98: 395–406.

Hurd, P.L., C.A. Wachtmeister and M. Enquist. 1995. Darwin’s principle of antithesis revisited—a role for perceptual biases in the evolution of intraspecific signals. Proceedings of the Royal Society of London Series B-Biological Sciences 259: 201–205.

Ison, M.J. and R.Q. Quiroga. 2008. Selectivity and invariance for visual object perception. Frontiers in Bioscience 13: 4889–4903.

Johnstone, R.A. 1994. Female preference for symmetrical males as a by-product of selection for mate recognition. Nature 372: 172–175.

Kamo, M., T. Kubo and Y. Iwasa. 1998. Neural network for female mate preference, trained by a genetic algorithm. Philosophical Transactions of the Royal Society of London Series B-Biological Sciences 353: 399–406.

Lehongre, K. and C. Del Negro. 2011. Representation of the bird’s own song in the canary HVC: Contribution of broadly tuned neurons. Neuroscience 173: 93–109.

Lewicki, M.S. and M. Konishi. 1995. Mechanisms underlying the sensitivity of songbird forebrain neurons to temporal order. Proceedings of the National Academy of Sciences of the United States of America 92: 5582–5586.

Lipin, M.Y., R.G. Smith and W.R. Taylor. 2010. Maximizing contrast resolution in the outer retina of mammals. Biological Cybernetics 103: 57–77.

Lledo, P.M., A. Saghatelyan and M. Lemasson. 2004. Inhibitory interneurons in the olfactory bulb: From development to function. Neuroscientist 10: 292–303.

Martin, K.A.C. 1994. A brief history of the feature detector. Cerebral Cortex 4: 1–7.

McClintock, W.J. and G.W. Uetz. 1996. Female choice and pre-existing bias: Visual cues during courtship in two Schizocosa wolf spiders (Araneae: Lycosidae). Animal Behaviour 52: 167–181.

Phelps, S.M. 2007. Sensory ecology and perceptual allocation: new prospects for neural networks. Philosophical Transactions of the Royal Society B-Biological Sciences 362: 355–367.

Phelps, S.M. and M.J. Ryan. 2000. History influences signal recognition: neural network models of tungara frogs. Proceedings of the Royal Society of London Series B-Biological Sciences 267: 1633–1639.

Rubner, J. and K. Schulten. 1990. Development of feature detectors by self-organization—a network model. Biological Cybernetics 62: 193–199.

Theunissen, F.E., N. Amin, S.S. Shaevitz, S.M.N. Woolley, T. Fremouw and M.E. Hauber. 2004. Song selectivity in the song system and in the auditory forebrain. In Behavioral Neurobiology of Birdsong (Zeigler, H.P. and P. Marler, eds). New York: New York Acad Sciences. pp. 222–245.

Tinbergen, N. 1948. Social releasers and the experimental method required for their study. Wilson Bulletin 60: 6–52.

Tinbergen, N. and A.C. Perdeck. 1950. On the stimulus situation releasing the begging response in the newly hatched herring gull chick (Larus argentatus argentatus Pont). Behaviour 3: 1–39.

Wilczynski, W., A.C. Keddyhector and M.J. Ryan. 1992. Call patterns and basilar papilla tuning in cricket frogs. 1. Differences among populations and between sexes. Brain Behavior and Evolution 39: 229–237.

Yang, X.L. and S.M. Wu. 1991. Feedforward lateral inhibition in retinal bipolar cells—input output relation of the horizontal cell depolarizing bipolar cell synapse. Proceedings of the National Academy of Sciences of the United States of America 88: 3310–3313.

Zhang, A.J. and S.M. Wu. 2009. Receptive fields of retinal bipolar cells are mediated by heterogeneous synaptic circuitry. Journal of Neuroscience 29: 789–797.

10.3 Rich Media Examples of Ritualization

Introduction

As discussed in Chapter 10, displays evolve from a wide variety of sender precursors. These precursors are usually cues that provide useful information to receivers that attend to and associate them with something the sender might do next. If the sender benefits from transmitting this information, selection can favor ritualization of the cue into a signal. Signal precursors include basic survival behaviors such as foraging, grooming, fleeing, and fighting, as well as physiological processes involved in growth, maintenance, and reproduction. Numerous examples were given in the text, and here we provide video and audio examples of some of them. This Web Topic unit is divided into sections based on type of precursor activity: comfort and hygienic activities, locomotory and feeding appendage movements, intention movements, motivational conflict, physiological processes (autonomic and endocrine systems), antipredator defenses, and a few interesting oddities.

Comfort and hygienic movement precursors

Many species of animals have ritualized body hygiene actions into signals. One of the first taxa in which such ritualization was recognized contains the ducks and geese (family Anatidae). In ducks, nearly all of their courtship and mating displays can be traced to precursors with more mundane functions. Many duck displays appear to be ritualized versions of “comfort” movements: behaviors such as shaking to remove water and arrange the feathers, preening, and using the head and beak to waterproof the plumage with oils produced by the uropygial gland. In addition, ducks have ritualized many other activities; other precursors of signals include drinking, tipping up to feed, leaping into flight, swimming on the surface, and diving. Many species are quite vocal and have added conspicuously stereotyped movements to sound production during courtship. In blue-winged ducks such as shovellers, the main male courtship display is a re-ritualized version of another signal. Ducks thus provide a rich panoply of ritualized signals.

Ducks are a very ancient group of birds and their higher-level taxonomy remains disputed. However, certain subfamilies are clearly distinct units with most members performing homologous courtship and mating displays. Some species might show all the displays known in its subfamily, whereas others might have exaggerated one or more displays and dropped the others. It is relatively easy to identify the same ritualized display in related species: they may differ slightly, but they are largely conserved during speciation. For this reason, duck displays were one of the first behavioral traits to be used for taxonomic classification (Johnsgard 1965; Lorenz 1971). Below, we provide a sampling of courtship displays in two different subfamilies of ducks. This provides a useful perspective on how ritualized displays can be conserved within a taxon, but still allows for species differences.

Dabbling ducks (Anatinae):Most of the species in this group belong to the genus Anas. The ancestors of the Anatinae must have had a large display repertoire since some species show most of the displays known in the subfamily, whereas other species show only a subset, with different species showing different subsets. Mallard ducks (Anas platyrhynchos) and speckled teal (Anas flavirostris) exhibit most of the shared repertoire. This includes ritualized drinking, an action shown by both sexes but often by males that is used in a wide variety of social contexts (see Figure 1.4 in Chapter 1). Ritualized and stereotyped female displays include inciting (in which the female accompanied by one male points her bill repetitively at another male, often inducing the first male to chase or attack the second); nod swimming (in which the female encourages males to display by stretching her neck forwards and swimming rapidly through the males in large circles); and head pumping (in which the female pumps her head up and down in synchrony with a male just prior to copulation). Male displays include preliminary shakes (ritualized shaking that may include wing flaps to announce that other displays will follow); grunt whistles (a rapid forward arching of the body, emission of a whistle, and the sweeping up of a stream of water droplets towards the rear with the bill; a major courtship display); head up–tail up (sudden stretching of head and neck upwards while also sticking the tail up vertically; another major courtship display); burping (a high pitched sound emitted while conspicuously stretching the neck and upwards or in a rotary fashion); down–up (rocking forward onto the breast and back; a signal most often directed at competing males); head pumping (male part of synchronized movement prior to copulation; male often dips bill at low point); bridle (opposite of down–up in that male rocks back on tail while lifting breast out of water; usually performed after a copulation); nod–swimming (same behavior as female but usually performed after a copulation).

Many of the clips below are digital versions of Super-8 movies taken from a water-level blind at the Bronx Zoo when one of the authors was a graduate student. Although the quality is low, the clips show displaying birds close up and are labeled with the name of the display and the species. A few more recent clips of mallards are included. Unfortunately, none of these clips include any corresponding sounds. In many of the clips, note the ritualized drinking interspersed between other displays:

• Female inciting: Female inciting is present in most Anatidae, not just the dabbling ducks. In this clip, examples in order of presentation are: Brazilean teal (Amazonetta brasiliensis), mallards, and common eiders (Somateria mollissima).
• Female nod–swim: The clip shows two nod–swims by a female mallard in the midst of many males.
• Male preliminary shakes: Again, a display more widely distributed in the Anatidae. In order of presentation: Bahama pintails (Anas bahamensis), garganey teal (Anas querquedula), marbled teal (Marmaronetta angustirostris), speckled (also called sharp-winged) teal (Anas flavirostris), and common eider.
• Male grunt–whistle: This striking display is limited to Anas species. Examples in order of presentation: green-winged teal (Anas crecca), mallards, and speckled (sharp-winged) teal.
• Male burping: Again, there is no sound accompanying these clips. In order of presentation, examples are: Baikal teal (Anas formosa), green-winged teal, Bahama pintail, red-billed duck (Anas erythrorhyncha), and garganey teal.
• Male Down–up and Head up–Tail up: In mallards, these tend to be given as separate displays. The down–up is often directed at other nearby males. In green-winged teal, a down–up is often, but not always followed by a head up–tail up. And in the Bahama pintail, which has lost the grunt–whistle display entirely, the main male courtship display is an obligate down–up/head up–tail up combination. Examples in order of presentation are green-winged teal, mallard, and Bahama pintails.
• Head pumping by both sexes: When a male and female are both interested in mating, they begin a synchronized head pumping that often culminates in copulation. Males often begin head pumping first, and females only join in if they are currently interested in mating:
  • Normal male–female examples: This clip shows a garganey teal male unsuccessfully trying to get a female to mate, followed by several mallard interactions, the last of which lead to a copulation.
  • Male head pumping as courtship display: Male blue-winged ducks such as shovellers (Anas clypeata) have co-opted the synchronized head pumping used by other species prior to copulation into their main courtship display; males swim around head pumping all the time. They have lost all of the other traditional dabbling duck displays such as grunt whistles, down–ups, and head up–tail ups but have added one novel display: ritualized mock feeding.
• Male bridle: This is usually performed by male dabbling ducks after copulations. Examples here are mallard and spectacled teal.
• Male nod swim: This is also performed by males after a copulation. Examples here are of mallards.

Stiff-tailed ducks (Oxyurinae): This small group of species has a pooled display repertoire that is quite different and surely evolved independently from that of dabbling ducks. As their name implies, males have a small rosette of stiff tail feathers that can be erected vertically when males are displaying. Males concurrently stretch their necks and raise their head into a stiff position and then perform some action ritualized from other types of movement: preening the breast, shaking, or swimming. Examples:

• Common ruddy duck (Oxyura jamaiecensis): The main male display is called the bubble display. Once the tail and head are vertically stretched upwards, the male tips his bill down and raps his breast repeatedly producing a froth of bubbles. It then sits stiffly in the vertical position before repeating the actions.
• White-headed ruddy duck (Oxyura leucocephala): This male performs a ritualized swimming next to the female with its bill aimed at the female. It then shifts into the head and tail erect position of most stiff-tailed ducks but holds this position instead of performing further actions. (http://www.arkive.org/white-headed-duck/oxyura-leucocephala/video-09a.html)
• Argentine ruddy duck (Oxyura vittata): Males of this species (which are notorious for having one of the longest penises of any animal), display by first assuming the vertical head and tail position common to congeners. Then, instead of rotating just the head to strike its breast repeatedly like the common ruddy, these males rock their entire body forwards and backwards in a violent motion. The amplitude of the rocking gets stronger as the display progresses.

Locomotory and feeding precursors

Drumming displays in woodpeckers: The primary signal for both territorial defense and mate attraction in most woodpeckers is a mechanical drumming sound. Woodpeckers tap on branches as they foraging and excavate holes for nesting and roosting. In the process of foraging, they encounter hollow branches that produce a loud resonating sound when tapped. The advertisement sound is a ritualized series of taps, either rapid drumming or a stereotyped pattern of tapping on various resonant structures. Examples of both types are included here:

• Yellow-bellied sapsucker (Sphyrapicus varius):
  • This clip shows the remarkable chiseling skill of a foraging bird as he carves off squares of bark to expose the tree’s sap. This is the precursor activity (http://www.arkive.org/yellow-bellied-sapsucker/sphyrapicus-varius/video-00.html).
  • This is an audio track of the sapsucker’s ritualized and patterned drumming sound when signaling (http://macaulaylibrary.org/audio/87149).
• Downy woodpecker (Picoides pubescens): This is a sound clip of a downy woodpecker’s drumming display  (http://macaulaylibrary.org/audio/44905).

Flight sounds in insects: The wings of insects often produce a buzzing or humming sound during flight. In a few cases, this sound plays a role in mate attraction.

• Yellow fever mosquito (Aedes aegupti): The flight tones of these mosquitoes are positively correlated with body size; the female fundamental frequency is around 450 Hz whereas the male’s frequency is 600–700 Hz. An interacting receptive female and male often shift their flight frequencies to match that of the other, but it is usually the second harmonic of the female and the third harmonic of the male that show the convergence at around 1200 Hz (Cator et al. 2009; Cator et al. 2010). This video provides some background and shows an example of a courting pair: (https://www.youtube.com/watch?v=Wgzr7ODahcA).

Stridulation and other mechanical sounds from wing and leg movements: We provide numerous examples of such sound signals that are ritualized versions of leg and wing movements in Web Topic 2.6.

Intention movements

As discussed in Chapter 10, the cues generated by animals preparing to adopt some action are prime precursors for ritualization into signals. Below we provide examples of several common categories of ritualized intention movements:

Intention to attack threats often display weapons and readiness to bite or lunge but may also be derived from non-aggressive precursors:

• Male white rhinoceros (Ceratotherium simum): Note curled tails and stiff-legged approach with heads down before actual fighting (http://www.arkive.org/white-rhinoceros/ceratotherium-simum/video-12b.html)
• Leopard cat (Prionailurus bengalensis): This clip shows a classical aggressive threat with teeth bared and ears back (http://www.arkive.org/leopard-cat/prionailurus-bengalensis/video-13.html)
• Gelada baboon (Thereopithecus gelada): In baboons, yawning, raising the eyebrows, and blinking have each been ritualized into threat displays. Olive and chacma baboons perform stylized yawns as threats; gelada baboons pull their lips back from their teeth. Partially closing the eyes to show the upper eyelids is also a threat. Some species raise their eyebrows or have special coloration of the upper lids to accentuate this display. This clip shows several gelada baboon males threatening using both raised lip and eye-flash displays. (http://www.arkive.org/gelada/theropithecus-gelada/video-12b.html)
• Male impalas (Aepyceros melampus): Males mark trees, open mouths, and present horns before engaging in sparring (http://www.arkive.org/impala/aepyceros-melampus/video-12.html)
• Male hooded seals (Cystophora cristata): Males of this species inflate a black sac on the forehead and a red nasal sac while vocalizing as threats to other males before beginning combat (http://www.arkive.org/hooded-seal/cystophora-cristata/video-12.html)
• Male northern elephant seals (Mirounga angustirostris): These males display their body sizes by rearing up and vocalizing before fighting. (http://www.arkive.org/northern-elephant-seal/mirounga-angustirostris/video-12.html)
• Ritualized fighting in male adders (Vipera berus): As with the seals, these snakes display their height and strength before actually engaging in combat. (http://www.arkive.org/adder/vipera-berus/video-11.html)

Submission and retreat signals are often opposite in form to attack signals:

• Crocodile (Crocodylus porosus): Whereas males lower their heads before attacking, an individual that is losing a fight raises its head in submission. (http://www.arkive.org/saltwater-crocodile/crocodylus-porosus/video-12a.html)
• Barbary macaques (Macaca sylvanus): Lip-smacking and teeth chattering are common submission signals in macaque monkeys. The teeth chattering shown here in Barbary macaques is used as a submissive signal, but is also used to indicate appeasement, affiliation, and reassurance. (http://www.arkive.org/barbary-macaque/macaca-sylvanus/video-12a.html)
• Rhesus macaques (Macaca mulatta): The following clip shows a wide mixture of both aggressive and submissive facial gestures in another species of macaque monkey. As with the Barbary macaque, lip smacking and teeth chattering are largely submissive signals. So is an open-mouth with the teeth and gums exposed, and flattening the ears against the head. An open mouth with the teeth covered is however a threat. Staring at an opponent, flashing the colored upper eyelids, and raising the eyebrows is also a threat. (http://www.arkive.org/rhesus-macaque/macaca-mulatta/video-12a.html)
• Chameleons (Furcifer pardalis): Aggressive and submissive displays are shown. Note the lateral compression broadside display and bright colors in the dominant animal and the evasive behaviors and pale colors in the subordinate. (http://www.arkive.org/panther-chameleon/furcifer-pardalis/video-12b.html)
• American bison (Bison bison): Courtship and aggressive behavior in bison; note the wallowing, calling, parallel walk, and head down aggressive threats, and the backing away submissive display by the loser of the fight. (http://www.arkive.org/american-bison/bison-bison/video-bi09a.html)

Other intention signals:

• Gerenuk (Litocranius walleri): The intention to mate display in many African antelopes involves the male tapping the inside of the female’s rear legs with his foreleg. Here we see this behavior in a pair of gerenuks (see also Figure 7.2, Chapter 7). (http://www.arkive.org/gerenuk/litocranius-walleri/video-09.html)
• Wolf spider (Pardosa nigriceps): Courtship is dangerous for male spiders as females are prone to eat them if they do not recognize the male as a conspecific. Males carry their sperm in their palps, which in this species are colored black. The male approaches the female waving the black palps in conspicuous but stylized ways to persuade the female that he intends to mate, not attack her or serve as prey. (http://www.arkive.org/wolf-spider/pardosa-nigriceps/video-09a.html)
• Herring gull (Larus argentatus): To initiate breeding, a mated pair must decide on a nest site. When one bird has found a possible site, it performs the choking display, a repeated downward pointing of the bill. If the mate doesn’t approve, they point at other spots until both choke, which indicates agreement. This clip is from the movie Signals for Survival (Cornell Lab of Ornithology).
• Canada geese (Branta canadensis): Geese, swans, and ducks usually synchronize taking flight by performing head-flicking intention movements before hand. These are likely ritualized and incomplete versions of the initial jump into flight that these heavy birds must perform to get off the ground. Here two Canada geese perform their version of this intention display. (http://www.youtube.com/watch?v=6pbCr577RI0&feature=related)

Motivational conflict precursors

In Chapter 10 (pp. 382–4), we discuss various mechanisms by which motivational conflict in animals might serve as a precursor to ritualized signals. Below, we provide a video clip of a ritualized signal that is thought to have arisen from each of the motivational routes mentioned in the text:

Ambivalence behaviors:

• Three-spined stickleback (Gasterosteus aculeatus): The zigzag courtship dance of the male stickleback is considered a classical example of a signal arising from motivational conflict. This video shows the entire nest-building and courtship sequence of which the zigzag dance is a part. (http://www.youtube.com/watch?v=cBX8hWuiHTk)

Broadside threats:

• Sable antelope (Hippotragus niger): Two males approach each other and the dominant performs a broadside threat that causes the subordinate to leave. (http://vimeo.com/22088612)

Displacement acts:

• Laughing gulls (Leucophaeus atricilla): In this clip, two individuals are contesting dominance status. Until the end, neither attacks the other, but instead bends over and pecks at the ground as if feeding. This is interpreted as a displacement act that has become ritualized into a signal of threat. (http://www.youtube.com/watch?v=9v4cmeoHcqE)

Redirected acts:

• Grass pulling in gulls may be a redirected act of aggression during fights. The movement is similar to the behavior of grasping the opponent’s bill or wing and pulling. This clip is from the movie Signals for Survival (Cornell Lab of Ornithology).

Motivation-structure rules in birds and mammals: As discussed in Chapter 10, the combination of multiple motivations and corresponding physiological states of actions appropriate to those motivations can combine in complicated ways to limit the kinds of sounds an animal might produce (see Figure 10.14). These favor higher frequencies as fear increases, and higher bandwidths as aggressiveness increases. Below, we provide some examples that appear to fit these motivation-structure rules:

• Bonobos (Pan paniscus): Threat calls of this species tend to be low frequencies whereas fearful screams are high frequency. (http://www.arkive.org/bonobo/pan-paniscus/video-13.html)
• Red deer (Cervus elaphus): Males with different ages, sizes, and motivations: produce roars at different dominant frequencies. Here are two examples. (http://www.arkive.org/red-deer/cervus-elaphus/video-13.html)
• Domestic dog (Canis lupus): Growls with different bandwidths and dominant frequencies produce different responses in receivers. (http://www.youtube.com/watch?v=fJI9QkNlLrc)
• Wart hogs (Phacochoerus africanus): Relaxed wart hogs tend to make low frequency grunts. When fighting, males alternate low frequency threats before a grapple, but then grunts and high frequency squeals during one. (http://www.arkive.org/common-warthog/phacochoerus-africanus/video-12.html)
• Carolina wren (Thryothorus ludovicianus): Calls of this species fit nicely within a motivation-structure rule scheme. In the table below, aggression increases as one moves to the right, and fear as one moves down. Click on the call name to hear the sound:

The calls in the first column (A–G) are tonal and relatively high in frequency. Calls become more harsh and broadband across the rows from left to right. The aggressive endpoint is the growl (C) often given before an attack, and the fear endpoint is the repeated pee, an appeasement call. The pi-zeet (A) is much like a contact call given to locate family members. The rasp (B) is given during mobbing and intraspecific fights. The chert is associated with movement and is highly variable in pitch, repetition rate, and repetition pattern, being lower and harsher when moving toward an opponent and higher and more tonal in submissive contexts. The cheer (E) is a male-only call sometimes given instead of a song during countersinging and sometimes during mobbing; females and juvenile males give the dit call during fights and before roosting. The chatter is a female-only aggressive call that sometimes overlaps her mate’s song and is directed at neighboring females. The nyerk (H) is also a female-only aggressive call often preceding the chatter. Scees (I) are distress calls given by birds that are losing a fight. (After Hagerty & Morton (1995); Owings & Morton (1998); recordings courtesy of Eugene Morton.)

Physiological processes

Piloerection: Autonomic processes such as erection or puffing of hair and feathers for thermoregulation have been ritualized by many birds and mammals, giving them voluntary control for display:

• Belted kingfisher (Megaceryle alcyon): Many birds have feathered head crests that they can raise or lower at will. Examples include jays, parrots, hoopoes, and kingfishers. While this clip is largely focused on the fishing skills of a belted kingfisher, it also shows the bird repeatedly raising and lowering its crest. (http://www.arkive.org/belted-kingfisher/megaceryle-alcyon/video-00.html)
• Superb lyrebird (Menura novaehollandiae): Clip begins with a male’s enormous tail plumage collapsed. Once it begins displaying, it erects the ornate tail feathers into a particular configuration while mimicking the calls of many other local species of birds. (http://www.arkive.org/superb-lyrebird/menura-novaehollandiae/video-13.html)
• Coyote (Canis latrans): Many mammals alternately sleek or fluff the fur on their bodies during dominance interactions and overt aggressive interactions. Fluffing of the fur over the whole body is usually associated with fear and defensive motivations (e.g., cats). In this clip, a group of coyotes fights over a carcass. Note how the degree of pelage fluffing or sleeking varies with the relative level of dominance. (http://www.arkive.org/coyote/canis-latrans/video-08d.html)

Blood flow: Vasodilation of relevant tissues is a common autonomic mechanism for thermoregulation or preparing tissues for flight or other action. A number of animal species expose patches of bare skin that are highly vascularized. Increasing blood flow into this skin colors it red. Such patches may be inflatable in various ways to produce large red display structures. Whereas many animals show red patches all the time, humans and a few other species can blush transiently and rapidly. Some animal examples with long-term red skin:

• Magnificent frigatebird (Fregata magnificens): Males have a large and inflatable throat sac that is highly vascularized and red. This clip shows the sac deflated when fighting with other males, and then inflated during display. (http://www.arkive.org/magnificent-frigatebird/fregata-magnificens/video-09a.html)
• Southern ground hornbill (Bucorvus cafer): Both sexes in this species have red vascularized tissues around the eyes and as part of an inflatable throat sac. (http://www.arkive.org/southern-ground-hornbill/bucorvus-cafer/video-09a.html)

Hormone-derived signals

• Domestic pigs (Sus scrofa): During courtship, males produce a frothy drool containing an androgen-derived pheromone; the male then nuzzles the female around the mouth and snout so she receives the pheromone, and eventually she stands still for mating. (http://www.youtube.com/watch?v=4tmmv7M0IAk&feature=related)

Anti-predator defenses as signal precursors

• Cotton leafworm moth (Spodoptera litura): Both sexes of this noctuid moth have ears tuned to detect the sonar calls of predatory bats. Males co-opt the tuned auditory sensitivity in females by producing courtship calls in the same ultrasonic range as the bat calls. (http://www.youtube.com/watch?v=E7kJYU_25To)
• Tiger moths (Arctiidae): Tiger moths not only detect the echolocation calls of predatory bats, but also produce ultrasonic clicks that can “jam” a bat’s sonar. Male tiger moths have co-opted this behavior by producing ultrasonic courtship sounds derived from their jamming clicks. This link takes you to the lab web site of Dr. William Conner. Select “Moth Movies,” and then view clips of Cycnia tenera, Empyreuma affinis, Syntomeida epilais, and Euchaetes bolteri. (http://college.wfu.edu/biolab/conner/moth-movies/)

Literature cited

Cator, L.J., B.J. Arthur, L.C. Harrington & R.R. Hoy. 2009. Harmonic convergence in the love songs of the dengue vector mosquito. Science 323: 1077–1079.

Cator, L.J., K.R. Ng’Habi, R.R. Hoy & L.C. Harrington. 2010. Sizing up a mate: variation in production and response to acoustic signals in Anopheles gambiae. Behavioral Ecology 21: 1033–1039.

Haggerty, T.M. & E.S. Morton. 1995. Carolina Wren (Thryothorus ludovicianus). In The Birds of North America Online (Poole, A., eds.). Ithaca: Cornell Lab of Ornithology. Issue 188.

Johnsgard, P.A. 1965. Handbook of Waterfowl Behavior. Ithaca NY: Comstock Publishing Associates. (Available online at: http://digitalcommons.unl.edu/bioscihandwaterfowl/7/)

Lorenz, K. 1971. Comparative studies of the motor patterns in Anatinae. In, Studies in Animal and Human Behaviour, Vol. II. (Translated by Robert Martin). Cambridge, MA: Harvard University Press. pp. 14–114.

Owings, D.H. & E.S. Morton. 1998. Animal Vocal Communication: A New Approach. Cambridge: Cambridge University Press.

10.4 Emotion, Drive, and Motivation

Introduction

Motivation theories attempt to explain how and why animals perform different behaviors at different times. The key principle underlying this book is that animals must continually make decisions about what activity to pursue given the variable nature of the social and ecological environment in time and space. This problem is particularly relevant for mobile animals, and in fact a centralized brain and conscious awareness of the environment are argued to have evolved as a consequence of the “liability” of mobility and the need to make constant decisions (Merker 2005). In this Web Topic unit we briefly review the development of motivation theories. Early models were not construed in ways that were readily testable by physiologists, but subsequent models steadily improved to enable such tests. Emotional constructs became embedded in this motivational framework. In some cases, emotions and motivation are signaled to receivers, or receivers may detect inadvertent cues from behaving individuals that reveal information about their motivational or emotional state. We therefore conclude this unit by discussing the relevance of these theories to the evolution of honest signaling.

“Drive” models of motivation

The first generation of motivational models viewed motivation as part of a homeostatic drive mechanism that energizes animals, pushing them to rectify physiological imbalances and thus satisfy their internal needs. Homeostasis, the tendency of a system to remain at a stable equilibrium, requires a regulatory system that maintains the animal’s physiological state at an optimal set point. If the physiological state moves off the set point, an error-detection mechanism triggers corrective responses to bring the animal back to the set point, like a thermostat (Cannon 1932; Hull 1943). The need to correct the imbalance generates the drive energy. This drive causes animals to respond to those stimuli that will satisfy the need and reduce the imbalance. Appropriate stimuli for the specific drive are learned and refined with experience, according to the response-reinforcement learning paradigm. In this view, response motor patterns that best meet the animal’s needs in a particular stimulus situation are selectively strengthened through reward reinforcement. Hunger and thirst drives seemed to fit this model well. If a deficit in nutrition or water is detected in the body, physiological mechanisms such as low blood glucose levels or body fluid levels initiate and direct food- or water-seeking behaviors. Once found, the animal consumes or drinks until the system set point is restored and the drive is reduced. Evolutionarily built-in goals, such as reproducing (sex drive) or chasing off a threatening rival (aggression drive), were similarly viewed as motivating set points from this homeostatic perspective. These ideas prompted researchers to search for the physiological set points and the deficit signals in the brain (Pfaff 1982). However, numerous problems emerged. First, set points appeared to be highly flexible and variable and thus not very stable (Bolles 1980). This observation lead to the alternative concept of settling points, a stable state caused by a balance of opposing forces, which gives the illusion of a set point but without the homeostatic mechanisms (Wirtshafter and Davis 1977). Second, an animal’s behavior could be strongly affected by anticipation of a goal, by social effects, and by availability of resources; in other words, other cognitive and external factors needed to be considered (Weingarten 1983; Schulkin 2003). Third, the homeostatic mechanism of drive reduction upon fulfilling the physiological need could not be demonstrated, even for hunger and thirst drives. For example, intraveneous feeding or introducing food directly into the stomach does not stop an animal or human from consuming more food (Miller and Kessen 1952; Turner et al. 1975).

Another early model was the hydraulic model of motivational drive by the ethologist Konrad Lorenz (Lorenz and Leyhausen 1973), which attempted to explain why the level of motivation builds up as a function of the time since the last reward. This model uses the metaphor of the flush toilet (Figure 1). Drive grows internally like the build-up of pressure in a fluid reservoir, which at some point bursts through an outlet. Proposed internal sources of the build-up could be physiological depletion cues or secreted hormones related to hunger, thirst, aggression, and sex. The model also incorporates the presence of external stimuli that can increase the probability of triggering the motivated behavior. An external stimulus is more likely to trigger the behavior if the drive pressure is high than when it is low. If the drive pressure is very high, it might burst through the outflow even with no external stimulus, called a vacuum response. This model was thus put forward as a way to explain the spontaneous occurrence of behavioral acts by animals that have been prevented from performing their normal behaviors. Although this model was not readily adopted by neuroscientists because it didn’t offer useful details of neural mechanisms, it did deal with the interaction between internal motivating factors and external stimuli in controlling behavior, and offered an explanation of the phenomenon of spontaneous acts by thwarted animals. Hogan (1997, 2005) developed a similar model of motivation around the concept of energy as a way to overcome some of the shortcomings of Lorenz’s model. He envisioned an energy variable that has multiple sources of internal and external input energy, and multiple means of dissipating the energy. This model still requires a threshold variable that specifies when the behavior is likely to occur.

The drive models explain some properties of motivated behaviors, such as the increase in goal-seeking behavior as a function of time since the last reward, the ability of animals to learn appropriate responses in a given stimulus situation, and the occasional examples of spillover behaviors, but they fail to provide a physiologically-based answer to the mysterious source of the driving energy, and don’t address the frequent observation of behavioral flexibility of responses by animals. Although they posit separate drive systems for different types of motivated behaviors, these models also don’t address the common observation of interactions between different drives, such as motivational conflict described in Chapter 10 of the main text (Berridge 2004; Anselme 2010).

Figure 1: Hydraulic drive model of motivation. Fluid enters a reservoir through the tap, which represents a continuing flow of energy from endogenous sources. The height of the fluid in the reservoir indicates the build-up of drive energy. The fluid is held in the reservoir by a spring-loaded valve, which represents the inhibitory pressure from higher-level brain areas. The weighted scale pan represents the strength of external releasing factors, which can increase the likelihood of valve release in addition to the pressure from the fluid in the reservoir. When the valve opens, energy flows through the outlet into a slanted trough. The holes on the bottom of the trough coordinate muscle action pattern responses, where the yellow numbers indicate the rank of the response strength. (After Lorenz 1950.)

“Incentive” models of motivation

In contrast to the energy-driven and drive-reduction models for behavioral decision making, incentive motivation models envision motivation as a mental state responsible for changing an animal’s receptiveness to specific environmental stimuli (Berridge 2004; Anselme 2010). The first step in the development of these ideas was the discovery that hungry rats would suddenly change their behavior when offered tasty (sweet) but nutritionless rewards (Pfaffmann 1960). Pleasurable sensations (called hedonic rewards) appeared to be motivating factors by themselves, even in the absence of a drive-reduction effect. Pleasurable sexual sensation, even the anticipation of it, was also found to be a powerful motivating force (Sheffield 1966). Thus affective reactions (i.e., emotions) such as liking, disliking, fear, and anger became more explicitly incorporated into this second generation of motivational models. Bolles proposed that animals were motivated by the learned expectation of hedonic reward, not by internal deficit states (Bolles 1972). Bindra then clearly laid out the alternate learning paradigm implicit in this view (1974, 1978). Rather than learn a specific motor pattern in response to a given rewarding (or aversive) stimulus, an animal’s response is dependent on what it perceives at the time of stimulus presentation. Via learning and experience, animals develop cognitive representations of correlated sets of environmental stimuli that characterize particular goals or incentive objects. The suite of stimulus characteristics representing these goals or objects then becomes strongly associated with the hedonic reward they provide. The objects thereby acquire an incentive salience value. Response acts are flexibly determined at the moment as a function of the animal’s current internal condition, motivational state, and sensory inflow from both within the body and from the environment. In contrast to the drive models, a physiological deficit does not drive the seeking behavior directly, but can magnify the hedonic impact and incentive salience of the object or conditioned stimulus associated with the goal (Toates 1986). Motivational states thus stimulate perception and focus attention on key eliciting stimuli in a given context so that the animal performs the appropriate set of behavioral acts to achieve its goals. These kinds of models account for observed flexibility by animals far better than the response-reinforcement associations postulated by the drive models. For example, animals often perform different responses to the same stimulus situation that nevertheless achieve the same goal. An animal may take advantage of a feeding or drinking opportunity, even though it is not particularly hungry or thirsty, because it is uncertain when such opportunities will occur next. And animals perform nuanced behaviors in response to complex social stimuli such as a rival or potential mate.

Incentive models predict that neural pathways for motivational resolution in the brain’s limbic area must project to higher brain areas responsible for sensory-perception integration of external stimuli, and not directly to motor areas as predicted by the drive models. Brain circuits and nuclei that fired in response to hedonic stimuli were soon discovered (Stellar 1982), and a new field called affective neuroscience emerged (Davidson and Sutton 1995; Panksepp 1998; Rolls 1999; Davidson et al. 2000; LeDoux and Phelps 2000). The search for drive-dedicated neurons initially found no evidence that specific motivations or emotions were restricted to specific neurons. Rather, stimulation of key regions of the forebrain basal ganglia and limbic system nuclei (hypothalamus, hippocampus, amygdala, ventral tegmentum, nucleus accumbens, and ventral pallidum) evokes a range of motivated behaviors and both positive and negative emotions. These brain regions are connected by circuits and feedback loops of neurons with different types of neurotransmitters that have excitatory, inhibitory, and disinhibitory effects (Figure 2). The mesolimbic dopamine system seems to be responsible for the arousal of non-specific “wanting” (Gray et al. 1999). Dopaminergic neurons originate in the ventral tegmentum of the basal forebrain and send projections to the hippocampus, amygdala, nucleus accumbens, and prefrontal cortex. Stimulation of the nucleus accumbens in particular initiates seeking or exploratory behavior (Reynolds and Berridge 2002). The prefrontal cortex is the site of executive function, where attention is focused on features of preferred stimuli and decision making is accomplished. Activation of the dopamine system then stimulates the release of the neurotransmitter acetylcholine by cholinergic neurons throughout the brain; acetylcholine facilitates the integration of sensory input in the cortex, and also operates at neuromuscular junctions in the brain stem to cause approach and retreat actions. GABAergic neurons send inhibitory signals among the ventral tegmentum, ventral pallidum, and specific areas within the core shell of the nucleus accumbens, and appear to play a major role in activating “liking” responses and positive emotional expressions (Berridge 2003). Complex feedback loops with other types of neurons may then disinhibit these inhibitory effects. Although these circuits and brain regions do not encode specific motivational systems, some evidence is accumulating for the existence of dedicated neuropeptide transmitters for some motivational functions. The hypothalamus appears to possess neurons containing dedicated hunger peptides (e.g., leptin, neuropeptide Y, cholecystokinin, ghrelin) and specific receptors for these peptides. Thirst may be regulated by brain receptors for another neuropeptide called angiotensin II. Endogenous opioid-peptides (e.g., endorphins, enkephalins, dynorphins, and endomorphins) play an important role in mediating positive emotions, liking, and reward-seeking. In the section below on emotions, we summarize the brain circuits involved in these and other motivational systems. The point here is that different forebrain circuits combine with cortical visual, auditory, and olfactory input in the prefrontal cortex to enable animals to take appropriate goal-directed actions, as predicted by the incentive models.

Figure 2: Brain areas involved in motivation and emotion. Sagittal section through a rat brain showing key nuclei in the limbic system. Main neuronal circuits for mesolimbic dopamine system, showing dopamine neurons ascending from ventral tegmentum (red), glutaminergic pathways (blue) linking prefrontal cortex, hippocampus, and basal ganglia, and GABAergic inhibitory neurons (green). Green boxes show key nodes in this system. Other nuclei and brain areas discussed in this web topic unit are also shown. Abbreviations: PAG = periaqueductal gray area; DR = dorsal raphe nucleus; BST = bed nucleus of the stria terminalis. (After Kelley and Berridge 2002; Berridge 2004.)

The next-generation incentive model, called the uncertainty processing theory (UPT), attempts to address three problems with the earlier incentive models: 1) how motivational state actually affects behavior and cognitive processing, 2) how motivational specificity can be achieved, and 3) how interactions and conflicts among different motivational systems are resolved. The UPT argues that motivation is the brain’s solution to the problem of environmental uncertainty about psychologically significant events (Anselme 2010). Motivation is proposed to operate like an information processing center that enables animals to acquire knowledge about the contexts in which important events are likely to occur, highlight such events or their stimulus features as incentive objects, and reduce uncertainty about them through the recruitment of anticipatory and attentional cognitive activity. The model, shown in Figure 3, is derived directly from current knowledge of motivational processing in the brain.

Figure 3. The uncertainty processing theory of motivation. The flow diagram shows neurotransmitter pathways to the anticipation processing center (yellow) and the attention processing center (light blue); type of neurotransmitter indicated in blue. An event important to the animal’s fitness is indicated by the red box. The animal’s psychological state of uncertainty about this event is represented by the orange box. The probability P_E of an event occurring ranges from 0 to 1; uncertainty is highest at P_E = 0.5; certainty is high if the event is either very likely to occur (P_E = 1) or very unlikely to occur (P_E = 0). Prior experience and knowledge of this event (purple) reinforces the incentive salience value (positive or negative valence) for this event. Excitatory and inhibitory connections among the basal ganglia, together with decisions in the prefrontal cortex, determine the type of motivation elicited by the event and initiate seeking behavior. Approach or avoidance depends on the direction of reinforcement in earlier encounters with the event. (After Anselme 2010.)

High uncertainty about especially salient events such as food items, mates, rivals, or predators, could imperil reproduction and survival, so striving to reduce uncertainty about both liked and disliked events is strongly favored by natural selection. Uncertainty is proposed to be the factor that releases motivated behaviors in changing environments. As mentioned earlier, mesolimbic dopamine plays a key role in causing reward expectation and wanting. The UPT goes one step further by suggesting that dopamine encodes uncertainty for a wide range of positive and negative events. Key evidence for this assertion derives from a study showing that activation of dopamine neurons peaks when the probability of receiving a reward is 0.5, and gradually diminishes as reward probability becomes more predictable (i.e., approaches 0 or 1) (Fiorillo et al. 2003). Moreover, dopamine neurons are more responsive to the anticipation of rewards rather than to the receipt of a reward, and to novel attention-grabbing stimuli (Schultz 1998, 2002). Dopamine projections to the prefrontal cortex, basolateral amygdala, and hippocampus are involved in anticipation of a reward. These areas in turn interact using glutamate and converge their projections onto the nucleus accumbens, which processes anticipation and facilitates flexible approach responses via indirect connections to motor neurons (Ikemoto and Panksepp 1999). In the next step, motivational specificity occurs via a selective attention process. Cholinergic neurons (acetylcholine) originating in the basal forebrain project to cortical regions of the brain and are involved in discrimination and processing of sensory information from all modalities. When an animal in a given physiological state (e.g., deprived) has the opportunity to establish physical contact with a useful object for that context (e.g., food), the object acquires psychological significance as a consequence of its hedonic value, which can then be represented mentally. The learned association between physiological state and the object’s value establishes a goal and modulates the seeking behavior. The response may be approach or withdrawal depending on whether the direction of reinforcement during prior encounters was positive or negative.

The UPT model offers a possible explanation for the behaviors animals perform when confronted with two conflicting events, as we described in the section on motivational conflict in Chapter 10 of the main text. Animals and humans presented with two types of salient stimuli show ambivalence, delayed reaction times, and increased error rates; if the responses are truly incompatible, one type of response eventually prevails while the other is inhibited (Figure 4A). This could be caused by the limited ability of the brain to attend to two different events at the same time, given the common pathway for anticipation and selective attention. The activation of the dopamine-cholinergic pathway for one event means that it is less available to deal with another event; the attention and response thresholds for this event are therefore raised (Figure 4B; see also main text Figure 10.7). If two events are both strong stimuli, they may each mutually inhibit the other as a consequence of attentional interference (Anselme 2010).

Figure 4: Attentional interference during motivational conflict. Two motivational causal factors are shown in both graphs with numbers indicating the different factors (e.g., thirst and hunger, or fear and anger). The attentional threshold is shown by the bar labeled A(1 or 2) and the higher response threshold by R(1 or 2); the bracketed M region indicates the motivational strength for the respective factor. (A) When only one motivational causal factor is stimulated to a level above its attentional and response thresholds, the attention and response thresholds for the other causal factor are raised (dashed arrows). The animal focuses it attention only on the strongly motivated stimulus and performs the appropriate response behavior, while attention and behaviors appropriate for the other causal factors are inhibited. (B) If two causal factors are both simultaneously stimulated, each raises the thresholds for the other one, but if the intensities of both stimuli are above their thresholds, the consequence is attentional interference (tan arrows), which in this case is stronger for M1 compared to M2. (After Anselme 2010.)

Whether or not dopamine turns out to encode event uncertainty, it clearly does instigate an urge to adopt resource-seeking behaviors such as approach, exploration, and investigation. In humans, dopamine injection causes euphoria. Which leads us to a discussion of the role of emotions in motivating behavior.

Emotions

Emotions are an important component of motivational systems. As discussed above, wanting is a dopamine-dependent process that makes animals more receptive to certain stimuli and transforms them into desired goals—the incentive salience component of motivation. The wanting system probably evolved early in vertebrate evolution to mediate the innate pursuit of key objects such as food and mates and the avoidance of stimuli associated with predators. Although desired goals are usually liked goals (and dangerous objects are fear-inducing objects), liking and disliking represent distinct hedonic (emotional) processes compared to wanting, and liking and wanting are facilitated by different neurotransmitters and neural circuits. Expressions of hedonic pleasure or displeasure and associated neurological changes occur upon consumption of a reward (e.g., tasty food versus bitter food). Such emotions operate as reinforcers during the process of learning the incentive value of various environmental stimuli (Cardinal et al. 2002; Berridge and Robinson 2003). Within the construct of motivation, the feelings generated by hard-wired emotional systems facilitate the acquisition of environmental knowledge and skill. In the broader view, different emotions stimulate organisms to respond quickly to external events in ways that optimize fitness (Darwin 1872).

Emotion is an umbrella term that includes affect (feelings), along with cognitive, behavioral, communicative, and physiological manifestations. Emotions are triggered by external events and often lead to rapid onset and short duration effects, in contrast to moods, which are brought on by less obvious antecedents and have longer lasting effects. Most mammals possess a core set of emotions—seeking, fear, anger, play, panic, lust, and care (Panksepp 2005). Birds and other vertebrates also show many of these emotional behaviors. Evidence for these core emotional systems includes the ability to elicit specific response behaviors by stimulating key subcortical brain nuclei (Panksepp 2003a), and in human fMRI studies the activation of different brain regions by individuals experiencing different emotions (Damasio et al. 2000; Phan et al. 2002). In humans, emotional responses occur on two levels: a subconscious level that involves subcortical pathways, the autonomic system, and involuntary behaviors and facial expressions; and a conscious level that directs the behaviors to appropriate targets and modulates some of the involuntary expressions. We don’t know the extent of animals’ cognitive experiences during emotional episodes but we do see evidence of the physiological effects, communicative expressions, and directed responses. Rats and monkeys will eagerly press levers to obtain an injected dose of heroin, a plant-derived opiate that mimics the pleasurable effects of endogenous opioids found in all mammalian species. Below is a brief overview of the core emotional systems. Each one is based on a different brain circuit that links basal ganglia, thalamus, and a distinct portion of the frontal lobe (Alexander and Crutcher 1990).

The seeking system is the generalized activation system recruited by all vertebrates to organize the pursuit of food, mates, and shelters. The energizing (wanting) component is driven by the dopaminergic-cholinergic pathway described earlier. Activation of the ventral tegmental area and nucleus accumbens plays a critical role in causing both approach and aversive action responses (Ikemoto and Panksepp 1999; Ikemoto 2007; Zellner and Ranaldi 2010). The hedonic (liking) component is caused by opioid neurons in the nucleus accumbens shell, ventral pallidum and brainstem. These neurons synthesize endogenous opioid peptide transmitters such as endorphins and enkephalins, as well as the specialized G protein receptors for these neurotransmitters. Stimulation or activation of these regions is responsible for the pleasurable sensations that occur once a tasty food item has been consumed (Berridge 2003). In rats, primates, and human infants, the sensation of sweet taste (from taste bud sensors) triggers a characteristic tongue-protruding response, while bitter taste triggers gaping (or the disgust facial expression in older humans). The wanting and liking circuits of the seeking system can be separately knocked out without destroying the other component. Together, the pleasurable experience of an object reinforces the subsequent motivated pursuit of similar objects. A good communication-related example of this seeking and rewarding system has been described in songbirds, where distinct patterns of dopamine activity influence the motivation to produce song, and opioids released as part of social interactions induce further singing (Riters 2011).

The fear system prepares an animal to take appropriate rapid actions in dangerous contexts. It has a dedicated circuit of excitatory (acetylcholinergic and epinephrinergic) neurons. The neural response is initiated in the lateral and central amygdala upon sensory detection of learned or innate cue features. The amydgala plays a key role in distinguishing between aversive and positive stimuli (Etkin et al. 2006; Tye and Janak 2007; Shabel and Janak 2009; Bermudez and Schultz 2010; Morrison and Salzman 2010; Tye et al. 2011). If the stimulus is deemed dangerous, a cascade of involuntary responses is mediated by nerve projections to the anterior and medial hypothalamus, on to the periaqueductal gray area (PAG) of the midbrain, and then to the lower brain stem and spinal cord (Panksepp 2004). This circuitry stimulates the sympathetic nervous system (a component of the autonomic nervous system) and triggers the release of the hormone epinephrine (also called adrenaline) from the adrenal gland, which targets various organs and causes an increase in heart rate, blood pressure and perspiration while inhibiting other non-essential body functions. The behavioral response to stimulation of these specific subcortical brain areas is freezing, startling, or fleeing; cognition may be involved in assessing which of several alternative escape behaviors is best given the current conditions. This emotion is sometimes accompanied by a scream vocalization, and in humans by a fearful facial expression.

The context and function of the remaining core emotions is the mediation of social interactions. The anger system shares some circuits with the fear system. The orbital frontal cortex first processes sensory input from social cues and signals, typically olfactory in the case of rodents and visual and vocal in the case of primates. Neural pathways then descend to the medial amygdala, which, as in the fear system, evaluates whether the stimulus is aversive or positive. Pathways then extend to several areas in the hypothalamus, thalamus, lateral septum, and bed nucleus of the stria terminalis. These centers send projections to the periaqueductal gray area and the lower brain stem, as in the fear system. The hypothalamus also connects directly to the pituitary gland, which releases various hormones directly into the body’s circulatory system that target the adrenal gland. Noradrenaline (norepinepherine) plays an important role in aggression as both a neurotransmitter in the brain and as a circulatory hormone produced by the adrenal gland. As in the fear response, these hormones increase heart rate and blood supply to the muscles and prepare an animal to fight if necessary. Various aggressive signals, facial expressions, and postures are involuntarily produced from the output of these subcortical ganglia. In addition, neural pathways from the subcortical ganglia ascend through the thalamus into the cortex to permit some voluntary control and assessment of the anger-evoking stimulus. The dopamine motivational circuit must be intact in order for animals to display the full aggressive behavioral repertoire toward a rival individual. Finally, steroid hormones such as testosterone and estrogen are also necessary to sustain aggressive behavior. Testosterone seems to primarily have an organizational effect on the brain, especially in males, by making aggression-inducing stimuli more salient; testosterone affects responsiveness in the lateral septum, amygdala, and dorsal raphe nucleus. Testosterone also promotes the development of aggressive behavioral skills through play behavior in young mammals (Nelson and Trainor 2007).

The play system also seems to have its own set of circuits, although they overlap with circuits for other emotions. Play behavior is the first type of non-mother-directed social interaction in a young mammal’s life. It provides a crucial opportunity for learning adult social skills, practicing aggressive behaviors, establishing one’s position in the dominance hierarchy of peers, and discriminating the sexes, and natural selection has imbued it with a pleasurable motivating reward system. The opioid and dopamine reward circuits are recruited to motivate play behavior. Cholinergic, noradrenergic, and opioid neuron circuits underlie the attentional processes needed for focusing on the rapidly performed actions and learning from errors. The cortex is not involved in play initiation, but it does affect play performance, in the sense that decorticated animals are hyperactive and very aggressive. Built-in mechanisms thus inhibit lethal acts during play. Playing animals also spontaneously give characteristic playful vocalizations. As mentioned above, testosterone is involved: males are more likely to initiate play and respond to play initiation signals by other males (Siviy and Panksepp 1987a, b; Panksepp et al. 1994; Vanderschuren et al. 1997; Knutson et al. 1998).

The panic system operates primarily in young mammals that have been separated from the parent, and elicits vocalizations of pain, distress, and crying. This behavior appears to be initiated in a region of the cortex called the anterior cingulate. Stimulation of this area leads to activation of the bed nucleus of the stria terminalis, the ventral septal and dorsal preoptic areas, and then on to the dorsomedial thalamus and the periaqueductal gray area of the brain stem. This circuit is called the thalamo-cingulate limbic pathway (Herman and Panksepp 1981; Panksepp 2003b; Newman 2007; Panksepp and Watt 2011). No learning is involved in this pathway, as it occurs in very young infants after being isolated. The effects can be modulated and diminished by the application of various opioid peptides, oxytocin, and the monoamine neurotransmitter serotonin.

Feelings of social attachment and bonding in mammals (care and lust systems) are facilitated by release of oxytocin and vasopressin, often called love hormones (Nelson and Panksepp 1998). These peptides are synthesized by specialized cells in the paraventricular nucleus of the hypothalamus. Some of these cells extend projections into other parts of the brain, such as the prefrontal cortex, basal ganglia (amygdala, ventromedial hypothalamus, septum, nucleus accumbens), and brain stem (Morgane et al. 2005), where they operate as neurotransmitters. Receptors for these neurotransmitters are also synthesized in these brain regions. Other cells in the hypothalamus send projections to the nearby pituitary gland, which stores oxytocin and vasopressin and releases these chemicals into the blood stream, where they operate as hormones. They target various organs involved in reproduction, including mammary glands and uterus in females and gonads in both sexes. Since these peptides are too large to pass the blood–brain barrier, their joint physiological and psychological effects are believed to be coordinated by synchronous release into the brain and circulatory system. Oxytocin is particularly important in female mammals. It increases immediately after birth and regulates aspects of bonding with infants (Nowak et al. 2007). A large surge of oxytocin occurs during sexual behavior and orgasm. In monogamous mammals such as the prairie vole (Microtus ochrogaster), administration of central oxytocin induces pair bond formation and greater social contact; this species has many more oxytocin receptors than closely related rodents with polygamous, non-pair-bond social systems (Williams et al. 1994). Vasopressin is chemically similar to oxytocin and plays a role in facilitating paternal care in male mammals. Some other neurotransmitters also facilitate positive social interactions. Endogenous opioids are rewarding and can induce odor and place preferences; they are also released during bouts of affiliative interaction such as suckling, physical contact, allogrooming, and social play. Opioids are postulated to encourage animals to engage in affiliative social behaviors by inducing a euphoric state (Nelson and Panksepp 1998). The opioid reward system also plays an important role in reinforcing sexual behavior in male mammals (Agmo and Berenfeld 1990).

An important message to be extracted from these summaries of the neurophysiological bases of emotional systems is that each is associated with diagnostic expressions—in other words, communication signals. Food-seeking is associated with expressions of liking, such as smiles and tongue extrusion, and expressions of disliking such as gaping and disgust facial expressions. Fear is associated with screams and fearful expressions; anger with aggressive postures, staring, weapon presentation, and mouth expressions; and panic with cries and expressions of pain and sadness. Care and lust systems are associated with physical contact gestures and smiles; and the play system is associated with invitation postures, play faces, and laughter. For the most part, these expressions occur involuntarily based on subcortical neural pathways to various motor systems; they can occur in animals with a non-functional cortex. They are therefore honest indicators of the emotional feelings the sender is experiencing. The signals have evolved for this purpose because of direct social benefits to senders and costs of cheating. Nevertheless, they can often be modulated and controlled voluntarily to some degree because of cortical loops in all of the systems. This issue is discussed in detail for human emotional expressions in Chapter 16.

A final point is the inevitable urge by biologists and psychologists to categorize emotions, especially in humans, because we have many more described emotions than animals do. Early attempts by several researchers converged on a two-dimensional model, in which emotions are placed on a grid or circle with two orthogonal axes, one representing level of pleasantness (e.g., positive versus negative feelings, or valence), and another axis representing arousal level (high versus low) (Russell 1980; Watson and Tellegen 1985; Thayer 1986; Russell et al. 1989; Larsen and Diener 1992; Yik et al. 1999). Figure 5 shows a melded version of these models.

Figure 5: Two dimensional circumplex model of emotions. The vertical axis is arousal level, and the horizontal level is pleasantness. Various emotions can be organized around the circle based on combinations of these two axes. (After Yik et al. 1999.)

This type of model was subsequently expanded in several ways. One idea was to greatly increase the number of divisions around the circle to 28, comprised of 14 bipolar pairs of emotions (calm–tension, certainty–uncertainty, compassion–anger, fun–boredom, pleasantness–unpleasantness, happiness–sadness, pleasure–pain, satisfaction–frustration, desire–reject, love–hate, courage–fear, strength–tiredness, enthusiasm–apathy, arrogance–humiliation). Opposing emotions in each pair are situated on opposite sides of the circle, and emotions are ordered around the circle so that similar emotions are adjacent; the earlier concept of two orthogonal axes, pleasantness and arousal, was also retained (Diaz et al. 2001). Another strategy was to add a third dimension. Figure 6 shows an example with somewhat similar ordering of categories around a circle, but with a third dimension depicting intensity (Plutchik 2001).

Figure 6: Three-dimensional circumplex model. The terms in black lettering around the circle represent the primary emotions. There are four pairs of primary dyads, with the contrasting type on the opposite side of the circle. The dyads are arranged so that similar emotions are adjacent, as in a color wheel. Colored regions represent gradations between the primary emotions. Emotional intensity is represented by the radius of concentric circles, folded to create a three-dimensional structure where intensity is the third dimension, as shown in the inset. This model does not reflect the orthogonal pleasantness and arousal axes of the earlier models. (After Plutchik 2001.)

The most recent models have used multivariate statistical analysis of a large number of variables describing different emotions, including appraisal, psychophysiological changes, motor expressions, action tendencies, subjective experiences, and level of emotional regulation and control to statistically sort out the number of dimensions needed to encompass all of the emotional terms. This type of analysis finds that three or four dimensions are required: hedonic valence, arousal, degree of control or power, and unpredictability (Laukka et al. 2005; Fontaine et al. 2007). One such model is illustrated in main text Figure 16.26. Although affective neurobiologists do not find these psychological models helpful, it could turn out to be the case that the emotional axes do correspond to levels of analysis of emotional contexts in the brain. Certainly the amygdala performs an initial analysis of valence (positive or negative), and dopamine and serotonin circuits then determine the motivation or arousal level. Emotions that are close to each other on the wheel may use more overlapping components of the neural circuits, and combination emotions may vary and balance the ratios of different neurotransmitters. As new techniques are developed for monitoring subtle neuronal and chemical changes in the brain in behaving animals, the differences and similarities among emotions may be better elucidated (Panksepp 2003a; Tye et al. 2011).

Literature cited

Agmo, A. and R. Berenfeld. 1990. Reinforcing properties of ejaculation in the male rat — role of opioids and dopamine. Behavioral Neuroscience 104: 177–182.

Alexander, G.E. and M.D. Crutcher. 1990. Functional architecture of basal ganglia circuits — neural substrates of parallel processing. Trends in Neurosciences 13: 266–271.

Anselme, P. 2010. The uncertainty processing theory of motivation. Behavioural Brain Research 208: 291–310.

Bermudez, M.A. and W. Schultz. 2010. Reward magnitude coding in primate amygdala neurons. Journal of Neurophysiology 104: 3424–3432.

Berridge, K.C. 2003. Pleasures of the brain. Brain and Cognition 52: 106–128.

Berridge, K.C. 2004. Motivation concepts in behavioral neuroscience. Physiology and Behavior 81: 179–209.

Berridge, K.C. and T.E. Robinson. 2003. Parsing reward. Trends in Neurosciences 26: 507–513.

Bindra, D. 1974. A motivational view of learning, performance, and behavior modification. Psychological Review 81: 199–213.

Bindra, D. 1978. How adaptive behavior is produced — perceptual-motivational alternative to response reinforcement. Behavioral and Brain Sciences 1: 41–52.

Bolles, R.C. 1972. Reinforcement, expectancy, and learning. Psychological Review 79: 394–409.

Bolles, R.W. 1980. Some functionalistic thoughts about regulation. In Analysis of motivational processes (Toates, T.W. and T.W. Halliday, eds.). New York: Academic Press. pp. 63–75.

Cannon, W.B. 1932. The Wisdom of the Body. New York: W.W. Norton and Company.

Cardinal, R.N., J.A. Parkinson, J. Hall and B.J. Everitt. 2002. Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neuroscience and Biobehavioral Reviews 26: 321–352.

Damasio, A.R., T.J. Grabowski, A. Bechara, H. Damasio, L.L.B. Ponto, J. Parvizi and R.D. Hichwa. 2000. Subcortical and cortical brain activity during the feeling of self-generated emotions. Nature Neuroscience 3: 1049–1056.

Darwin, C. 1872. The Expression of the Emotions in Man and the Animals. London: John Murray.

Davidson, R.J., D.C. Jackson and N.H. Kalin. 2000. Emotion, plasticity, context, and regulation: Perspectives from affective neuroscience. Psychological Bulletin 126: 890–909.

Davidson, R.J. and S.K. Sutton. 1995. Affective neuroscience — the emergence of a discipline. Current Opinion in Neurobiology 5: 217–224.

Diaz, J.L., E.O. Flores and F. Sagan. 2001. The structure of human emotions: A chromatic model of the affective system. Salud Mental 24: 20–35.

Etkin, A., T. Egner, D.M. Peraza, E.R. Kandel and J. Hirsch. 2006. Resolving emotional conflict: A role for the rostral anterior cingulate cortex in modulating activity in the amygdala. Neuron 51: 871–882.

Fiorillo, C.D., P.N. Tobler and W. Schultz. 2003. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299: 1898–1902.

Fontaine, J.R.J., K.R. Scherer, E.B. Roesch and P.C. Ellsworth. 2007. The world of emotions is not two-dimensional. Psychological Science 18: 1050–1057.

Gray, J.A., V. Kumari, N. Lawrence and A.M.J. Young. 1999. Functions of the dopaminergic innervation of the nucleus accumbens. Psychobiology 27: 225–235.

Herman, B.H. and J. Panksepp. 1981. Ascending endorphrin inhibition of distress vocalization. Science 211: 1060–1062.

Hogan, J.A. 1997. Energy models of motivation: A reconsideration. Applied Animal Behaviour Science 53: 89–105.

Hogan, J.A. 2005. Causation: the study of behavioural mechanisms. Animal Biology 55: 323–341.

Hull, C.L. 1943. Principles of behavior. New York: Appleton-Century-Crofts.

Ikemoto, S. 2007. Dopamine reward circuitry: Two projection systems from the ventral midbrain to the nucleus accumbens-olfactory tubercle complex. Brain Research Reviews 56: 27–78.

Ikemoto, S. and J. Panksepp. 1999. The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Research Reviews 31: 6–41.

Kelley, A.E. and K.C. Berridge. 2002. The neuroscience of natural rewards: Relevance to addictive drugs. Journal of Neuroscience 22: 3306–3311.

Knutson, B., J. Burgdorf and J. Panksepp. 1998. Anticipation of play elicits high-frequency ultrasonic vocalizations in young rats. Journal of Comparative Psychology 112: 65–73.

Larsen, R.J. and E. Diener. 1992. Promises and problems with the circumplex model of emotion. In Review of Personality and Social Psychology: Emotion (Vol. 13) (Clark, M.S., eds.). Newbury Park, CA: Sage. pp. 25–59.

Laukka, P., P.N. Juslin and R. Bresin. 2005. A dimensional approach to vocal expression of emotion. Cognition and Emotion 19: 633–653.

LeDoux, J.E. and E.A. Phelps. 2000. Emotional networks in the brain. In Handbook of Emotions (Lewis, M. and J.M. Haviland-Jones, eds.). New York: Guilford. pp. 157–172.

Lorenz, K. and P. Leyhausen. 1973. Motivation of Human and Animal Behavior: An Ethological View. New York: Van Nostrand-Reinhold.

Lorenz, K.Z. 1950. The comparative method in studying innate behaviour patterns. Symposia of the Society for Experimental Biology 4: 221–268.

Merker, B. 2005. The liabilities of mobility: A selection pressure for the transition to consciousness in animal evolution. Consciousness and Cognition 14: 89–114.

Miller, N.E. and M.L. Kessen. 1952. Reward effects of food via stomach fistula compared with those of food via mouth. Journal of Comparative and Physiological Psychology 45: 555– 564.

Morgane, P.J., J.R. Galler and D.J. Mokler. 2005. A review of systems and networks of the limbic forebrain/limbic midbrain. Progress in Neurobiology 75: 143–160.

Morrison, S.E. and C.D. Salzman. 2010. Re-valuing the amygdala. Current Opinion in Neurobiology 20: 221–230.

Nelson, E.E. and J. Panksepp. 1998. Brain substrates of infant–mother attachment: Contributions of opioids, oxytocin, and norepinephrine. Neuroscience and Biobehavioral Reviews 22: 437–452.

Nelson, R.J. and B.C. Trainor. 2007. Neural mechanisms of aggression. Nature Reviews Neuroscience 8: 536–546.

Newman, J.D. 2007. Neural circuits underlying crying and cry responding in mammals. Behavioural Brain Research 182: 155–165.

Nowak, R., M. Keller, D. Val-Laillet and F. Levy. 2007. Perinatal visceral events and brain mechanisms involved in the development of mother–young bonding in sheep. Hormones and Behavior 52: 92–98.

Panksepp, J. 1998. Affective Neuroscience: The Foundations of Human and Animal Emotions. Oxford: Oxford University Press.

Panksepp, J. 2003a. At the interface of the affective, behavioral, and cognitive neurosciences: Decoding the emotional feelings of the brain. Brain and Cognition 52: 4–14.

Panksepp, J. 2003b. Feeling the pain of social loss. Science 302: 237–239.

Panksepp, J. 2004. The emerging neuroscience of fear and anxiety: Therapeutic practice and clinical implications. In Textbook of Biological Psychiatry (Panksepp, J., eds.). Hoboken, NJ: Wiley. pp. 587–589.

Panksepp, J. 2005. Affective consciousness: Core emotional feelings in animals and humans. Consciousness and Cognition 14: 30–80.

Panksepp, J., L. Normansell, J.F. Cox and S.M. Siviy. 1994. Effects of neonatal decortication on the social play of juvenile rats. Physiology and Behavior 56: 429–443.

Panksepp, J. and D. Watt. 2011. Why Does Depression Hurt? Ancestral Primary-Process Separation-Distress (PANIC/GRIEF) and Diminished Brain Reward (SEEKING) Processes in the Genesis of Depressive Affect. Psychiatry-Interpersonal and Biological Processes 74: 5–13.

Pfaff, D.W. 1982. Physiological Mechanisms of Motivation. New York: Springer-Verlag.

Pfaffmann, C. 1960. The pleasures of sensation. Psychological Review 67: 253–268.

Phan, K.L., T. Wager, S.F. Taylor and I. Liberzon. 2002. Functional neuroanatomy of emotion: A meta-analysis of emotion activation studies in PET and fMRI. Neuroimage 16: 331–348.

Plutchik, R. 2001. The nature of emotions — human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice. American Scientist 89: 344–350.

Reynolds, S.M. and K.C. Berridge. 2002. Positive and negative motivation in nucleus accumbens shell: Bivalent rostrocaudal gradients for GABA-elicited eating, taste “liking”/“disliking” reactions, place preference/avoidance, and fear. Journal of Neuroscience 22: 7308–7320.

Riters, L. V. 2011. Pleasure seeking and birdsong. Neuroscience and Biobehavioral Reviews 35: 1837–1845.

Rolls, E.T. 1999. The Brain and Emotion. Oxford: Oxford University Press.

Russell, J.A. 1980. A circumplex model of affect. Journal of Personality and Social Psychology 39: 1161–1178.

Russell, J.A., A. Weiss and G.A. Mendelsohn. 1989. Affect grid — a single-item scale of pleasure and arousal. Journal of Personality and Social Psychology 57: 493–502.

Schulkin, J. 2003. Rethinking Homeostasis: Allostatic Regulation in Physiology and Pathophysiology. Cambridge, MA: MIT Press.

Schultz, W. 1998. Predictive reward signal of dopamine neurons. Journal of Neurophysiology 80: 1–27.

Schultz, W. 2002. Getting formal with dopamine and reward. Neuron 36: 241–263.

Shabel, S.J. and P.H. Janak. 2009. Substantial similarity in amygdala neuronal activity during conditioned appetitive and aversive emotional arousal. Proceedings of the National Academy of Sciences of the United States of America 106: 15031–15036.

Sheffield, F.D. 1966. New evidence on the drive induction theory of reinforcement. In Current Research in Motivation (Haber, R.N., ed.). New York: Holt, Rinehard, and Winston. pp. 111–122.

Siviy, S.M. and J. Panksepp. 1987a. Juvenile play in the rat — thalamic and brain stem involvement. Physiology and Behavior 41: 103–114.

Siviy, S.M. and J. Panksepp. 1987b. Sensory modulation of juvenile play in rats. Developmental Psychobiology 20: 39–55.

Stellar, E. 1982. Brain mechanisms in hedonic processes. In The Physiological Mechanisms of Motivation (Pfaff, D.W., eds.). New York: Springer-Verlag. pp. 377–408.

Thayer, R.E. 1986. Activation–deactivation adjective check list — current overview and structural analysis. Psychological Reports 58: 607–614.

Toates, F. 1986. Motivational Systems. Cambridge: Cambridge University Press.

Turner, L.H., R.L. Solomon, E. Stellar and S.N. Wampler. 1975. Humoral factors controlling food intake in dogs. Acta Neurobiologiae Experimentalis 35: 491–498.

Tye, K.M. and P.H. Janak. 2007. Amygdala neurons differentially encode motivation and reinforcement. Journal of Neuroscience 27: 3937–3945.

Tye, K.M., R. Prakash, S.Y. Kim, L.E. Fenno, L. Grosenick, H. Zarabi, K.R. Thompson, V. Gradinaru, C. Ramakrishnan and K. Deisseroth. 2011. Amygdala circuitry mediating reversible and bidirectional control of anxiety. Nature 471: 358–362.

Vanderschuren, L., R.J.M. Niesink and J.M. VanRee. 1997. The neurobiology of social play behavior in rats. Neuroscience and Biobehavioral Reviews 21: 309–326.

Watson, D. and A. Tellegen. 1985. Toward a consensual structure of mood. Psychological Bulletin 98: 219–235.

Weingarten, H.P. 1983. Conditioned cues elicit feeding in sated rats: a role for learning in meal initiation. Science 220: 431–433.

Williams, J.R., T.R. Insel, C.R. Harbaugh and C.S. Carter. 1994. Oxytocin administered centrally facilitates formation of a partner preference in female prairie voles (Microtus ochrogaster). Journal of Neuroendocrinology 6: 247–250.

Wirtshafter, D. and J.D. Davis. 1977. Set points, settling points, and the control of body weight. Physiology and Behavior 19: 75–78.

Yik, M.S.M., J.A. Russell and L.F. Barrett. 1999. Structure of self-reported current affect: Integration and beyond. Journal of Personality and Social Psychology 77: 600–619.

Zellner, M.R. and R. Ranaldi. 2010. How conditioned stimuli acquire the ability to activate VTA dopamine cells: A proposed neurobiological component of reward-related learning. Neuroscience and Biobehavioral Reviews 34: 769–780.

10.5 Evolutionary Game Theory

Introduction

In Chapters 9 and 10 of the text, we introduced the basic logic of evolutionary game theory. However, we did not provide any guidance on how to find evolutionary stable strategies (ESSs). It turns out that appropriate methods depend a lot on how the game is structured. In this Web Topic, we provide more details on the underlying logic of evolutionary game theory, outline a general schema for classifying evolutionary games, and then provide some examples of finding ESSs for several game categories. Because this field has grown very rapidly since Maynard Smith’s influential book in 1982, this Web Topic can at best serve as an introduction. Our goal here is to provide just enough background to make the discussions in the text understandable. At the end of the module, we provide references that, with varying degrees of accessibility, allow readers to go beyond the introductory material presented here.

Basic concepts and definitions

• Games versus simple optimality: When an animal adopts an optimal behavior to cope with some inanimate context, the latter will not change in response. For example, if an animal’s optimal behavior on a hot day is to move into the shade, the weather does not respond by snowing to spite the animal. It could snow, but not because the animal moved into the shade. However, this is not the case if the context in which the animal must find an optimal behavior involves other living creatures. Unlike the weather, living creatures are likely to change their physiology and behaviors according to what our focal animal decides to do. Usually, both parties seek to maximize their own payoffs despite the actions of the other party. The interaction of two or more animate parties each trying to select the optimal behavior given the behaviors of the others constitutes a game. Finding an optimal strategy in a game is clearly different from finding an optimal strategy in an inanimate context.

• Game equilibria: Each player in a game should adopt the best response to the behavior selected by its opponent. At first glance, this might seem likely to lead to an endless chain of moves and countermoves. However, if the players each hit on a best response to the other’s best response, the process will stop. At such a Nash equilibrium, it does not pay for either player to shift strategies (Nash 1950, 1951). Note that the players may have access to the same alternative strategies (a shared role) or different strategies (different roles). In either case, it is the combination of strategies adopted by the parties that defines the equilibrium. Both pure and mixed strategies (a fixed ratio of pure strategies) can lead to Nash equilibria. A Nash equilibrium can be strict, in that only one strategy for each party is its best response, or non-strict in that one or both parties can play any of several equivalent strategies and still maintain the equilibrium. Neither player need maximize their payoffs at a Nash equilibrium; in fact, either or both might do better at some other combination of player strategies. However, once both players hit on each other’s best response, it will not pay for either to shift to another strategy. Depending on the game, there can be no, one, or many possible Nash equilibria. In the latter case, which one is observed may depend only on chance and history.

• Evolutionary games: While classical economics focused on the optimal interactions of a particular pair of game opponents, evolutionary game theory shifts the emphasis from specific individuals to the relative success of alternative strategies in a population of animals. If animals that adopt a given strategy have higher fitness than those in the same role but who adopt alternative strategies, the former will spread in the population over evolutionary time. In a large and well-mixed population, animals playing one strategy encounter other animals playing the same or different strategies in proportion to their relative abundances. The average payoff for a given strategy is then the sum of the payoffs between it and all strategies currently in the population, each discounted by the current abundance of that strategy. Because this average payoff varies depending on the current mix of strategies in the population, evolutionary games are always frequency-dependent. Although played out by individuals within a generation (the “inner game” of Vincent and Brown [1988]), evolutionary games are basically multi-generational competitions between strategies (the “outer game”).

• Evolutionarily stable strategies: An evolutionarily stable strategy (ESS) is one that, when common, cannot be invaded by any rare alternative strategy (Maynard Smith and Price 1973; Maynard Smith 1982). This means that an ESS strategy is the best response both to itself and to any other rare strategy. If two ESSs are each the other’s best response, they generate a Nash equilibrium. The two ESSs will be the same strategy if the interacting animals have the same role. In this case, the Nash equilibrium can be strict in that only one pure strategy will be the ESS for all parties. Alternatively the equilibrium may be non-strict in that the ESS is some mix of strategies, any of which when given as part of the mixture or singly will provide equally good responses. If the interacting parties have different roles (e.g., males versus females or species X versus species Y) and the available strategies differ for the two roles, there will be two ESSs defining the equilibrium: one for each role. In this case, each ESS must be a pure strategy, and the corresponding Nash equilibrium will be strict.

• Roles and equilibria: Where two or more roles are present in a game, each role is likely to have its own strategy set. Where this is the case, any equilibria are called separating equilibria. Alternatively, if multiple roles adopt the same strategy set, a stable outcome is said to be a pooling equilibrium. It is also possible that some roles pool strategies whereas other roles each have their own strategy sets. Any equilibria in this situation are said to be semi-pooled. While these distinctions are not needed if there is only one role, or even for simple games with a few roles and a few strategy options per role, they do become useful in describing outcomes of games in which strategy sets are continuous and players may differ widely in available strategies. For example, in games describing optimal levels of male courtship display, males may differ in health or condition (role). If they each adopt different (honest) levels of display, any ESS would be a separating equilibrium; if those with less than average health display at one level and those of higher than average display at another, any ESS would be a pooling equilibrium. A semi-pooled equilibrium could occur if the less healthy males all adopted one display level, but above average males each used a display corresponding to its health.

• Equilibrium dynamics: The original definition of an ESS assumed haploid genetics, infinite population sizes, and sufficient “mixing” so that the relative abundances of the alternative strategies were identical everywhere. The definition also assumed that only one type of invading mutant was likely to occur at any time. This was in many ways a “static” definition in that it ignored whether and how strategies that originally appeared as rare mutants might increase in frequency until they arrived as dominant components of an ESS. The assumption that the population was infinite eliminated the effects of drift and thus any worries about what would happen if a population drifted off of an ESS equilibrium. Would it come back over time, or just wander away? And was an equilibrium that was resistant to invasion by a single mutant strategy necessarily resistant to multiple types of invaders at the same time?

  Such questions inspired subsequent research on the adaptive dynamics of evolutionary games instead of just focusing on the end-points (Taylor and Jonker 1978; Eshel 1983; Eshel and Feldman 1984; Hofbauer and Sigmund 1988; Vincent and Brown 1988; Abrams et al. 1993; Dieckmann and Law 1996; Metz et al. 1996; Eshel et al. 1997; Geritz et al. 1997; Geritz et al. 1998; Hofbauer and Sigmund 1998; Weibull 1998; Cresssman 2003; Vincent and Brown 2005; Nowak 2006; McGill and Brown 2007; Dercole and Rinaldi 2008; Apaloo et al. 2009; Apaloo and Butler 2009; Cressman 2009). These studies made it clear that the conditions for an ESS were insufficient to account for the many types of equilibria that might arise in evolving systems. They were not even sufficient for the persistence of ESS strategies in nature. In particular, two additional strategy properties are needed to fully describe all likely trajectories and equilibria.

  To visualize the significance of the three strategy properties, it is useful to think of adaptive landscapes (see Chapter 9). Normally, one envisions populations as moving up the slopes of an adaptive landscape towards regions of higher average fitness as the relative frequencies of alternative strategies shift due to selection and mutation. An equilibrium occurs when the population arrives at a location where the slope of the landscape is zero. Such an equilibrium can occur on a peak (a fitness maximum), in a pit (a fitness minimum), or along a valley (a saddle point where fitness increases along one axis but decreases along an orthogonal axis). Evolutionary games are more complicated than simple optimization processes because the shape of their adaptive landscapes can change as the frequencies of alternative strategies change (Eshel 1983; Abrams et al. 1993; McGill and Brown 2007). What starts as a population’s trajectory uphill can gradually change into a flat or even downhill trajectory as successive changes in population composition change the landscape. This greatly complicates the possible trajectories and affects the types of equilibria that might be encountered.

  The original definition of an ESS dodged the possibility that the underlying landscape might change by assuming that only a single type of mutant could appear at a time, and when one did, it was so rare that its presence did not change strategy frequencies or the adaptive landscape. If either condition failed to be met, the landscape could change, and then the strategy might no longer be uninvadable. Adaptive dynamic analyses indeed identified systems in which an ESS might not be sufficient for long-term stability in real populations. However, invoking a second and independent property of strategies, convergence stability, solves this problem. An equilibrium strategy is convergence stable if a mutant strategy closer to it in the adaptive landscape than the current mean of the nearby population can always invade (Eshel 1983; Abrams et al. 1993; Rand et al. 1994; Eshel et al. 1997; Geritz et al. 1998; Leimar 2005, 2009). Successive mutants would then move the population closer and closer until it reaches the equilibrium. Equilibrium strategies that are convergence stable are called “attractors.” How does this help? Unlike the assumption made to define ESSs, natural populations are finite and experience drift in strategy composition. Suppose a population at an ESS drifts to neighboring but different strategy values. Without convergence stability, the population will be under no selective pressure to move back. Convergence stability can both move a population to a nearby ESS in the first place and keep it there afterwards.

  An additional relevant property is neighborhood invadability (Geritz et al. 1998; Apaloo et al. 2009; Apaloo and Butler 2009; Cressman 2009). This is the ability of a strategy to invade neighboring strategies that are common even if it is rare. This property allows a mutant strategy that could, if common, be an ESS to gain a foothold in a population currently using other strategies. Note that as this mutant invades, it will change the relative frequencies of strategies in the population, and this could alter the adaptive landscape. Just because a strategy can invade when rare does not guarantee that it will be favored when more common. A strategy that is a good neighborhood invader in one context may or may not also be an ESS in another context.

  If strategies can be either ESS or not, convergence stable or not, and neighborhood invaders or not, there are eight possible combinations (Geritz et al. 1997; Geritz et al. 1998; McGill and Brown 2007; Apaloo and Butler 2009; Cressman 2009). Only six of these are biologically realistic, and when one controls for some dependencies between options, two combinations stand out as most relevant. Perhaps the most important combination is a strategy that is an ESS, convergence stable, and a good neighborhood invader. Such a strategy will move up to a peak in the adaptive landscape where it will become a stable equilibrium. It is now widely believed that the persistent strategies one sees in nature usually represent this combination. The second relevant combination is a strategy that is convergence stable but not an ESS. These usually occur as minima or saddle points in the adaptive landscape and can, at least theoretically, give rise to evolutionary branching in which the population splits into two groups with divergent trajectories. These cases may be one source for stable behavioral polymorphisms within a species or even speciation.

• Analytical formats: A variety of techniques have been developed for finding ESSs. Two of the more traditional approaches were already described on p. 330 and Figure 9.4 in the text. Below, we briefly review these again and contrast them with the adaptive dynamics approach:

• Extended form games: consist of a branching tree whose junctions represent possible alternative actions by nature or the players (Gintis 2000; Cresssman 2003; Hurd and Enquist 2005). The tree represents all possible paths that could be followed during the course of the game. Payoffs to each player are then provided at the end of each final branch. Extensive form games retain the greatest detail in a game’s structure. If a game requires sequential moves by players or nature, the cumulative process can be tracked in detail. It is also quite easy to examine the dynamics of sub-games (parts of the tree) in this format.

• Normal form: is the other traditional approach (Maynard Smith 1982). Here, the alternative strategies are listed in a matrix as rows for one player and columns for the other, and the payoff values for each combination of strategies are placed in the respective cells of the matrix. Collapsing an extensive form game into normal form eliminates most tracking and sub-game information. However, as we shall show below, one can often find ESSs solely with the normal form matrix; the details in the extensive form version of these games, while perhaps interesting, are often not essential to identifying stable equilibria. On the other hand, if the game consists of many successive and conditional choices of strategy, extensive form may be the only way ESSs can be identified.

• Adaptive dynamics: uses many of the tools and generalizations from dynamic systems analysis (Strogatz 1994). Instead of focusing on end points, adaptive dynamics analysis derives an equation for each strategy that predicts the change in that strategy’s abundance when it is played amidst the current abundances of each strategy in the population, the current size of the population, and the relevant fitness payoffs of each strategy combination (Taylor and Jonker 1978; Hofbauer and Sigmund 1998; Weibull 1998; Vincent and Brown 2005; Nowak 2006). If changes occur at intervals (e.g., no overlap of generations), the output of the relevant difference equation is the abundance of the strategy in the next time interval. If change is relatively continuous (e.g., overlap of generations and large populations), the output of the differential equation is the rate of change in the abundance of the strategy. If the population size is fixed or assumed to be infinite with good mixing, population size can be deleted from the arguments of the equation and the absolute abundances of strategies can be replaced by relative frequencies (i.e., fractions). There should be one such equation for each alternative strategy.

  Where the equations focus on relative strategy frequencies, one can visualize the resulting trajectories by placing them in a simplex (see Nowak 2006). This is an equilateral graph object with dimensions one less than the number of strategies being considered: two strategies have a single straight line for a simplex, and a three-strategy simplex is an equilateral triangle. Each point in the simplex represents a possible population mixture of the strategies: the vertices represent a single pure strategy, edges represent mixtures of two strategies, faces represent mixtures of all but one strategy, and the interior points are mixtures containing all strategies. One can begin analysis at a given point in the simplex and use the relevant equations iteratively to find successive points in the evolutionary trajectory. Equilibria occur when the trajectory reaches a point at which the equations all predict zero change in strategies in the next time interval. The type of equilibrium (fitness peak, fitness minimum, or saddle point) and whether the equilibrium is convergence stable or not can be determined by examining higher order derivatives of the equations. Where the mathematics are too complicated for analytical results, one can use simulations beginning from many different initial points to characterize likely trajectories, find equilibria, and identify their stabilities.

  A variety of dynamic equations have been used in this manner. One, the replicator equation, focuses on relative frequencies and ignores mutation and population size (Taylor and Jonker 1978; Hofbauer and Sigmund 1988, 1998; Nowak 2006). It equates per capita changes in strategy frequencies with the difference between the fitness of each strategy and the population average fitness. Though seemingly simple in general form, it has proven to have considerable flexibility and utility in many specific applications. The canonical equation of adaptive dynamics is very similar but replaces differences in fitness between an animal adopting a given strategy and the current population mean with derivatives of fitness versus strategy in its arguments (Dieckmann and Law 1996). Other approaches such as G-function equations predict fitness and use derivatives of the G-function to monitor change. These equations often include strategy abundances, resource availability, and other factors as arguments. This facilitates the application of evolutionary game theory to multiple species contexts in ecology (Vincent and Brown 1988; Vincent and Brown 2005; McGill and Brown 2007).

A general schema for classifying evolutionary games

There are six independent criteria that one can invoke to classify any evolutionary game. Different combinations of values for these criteria define different game categories. Each category tends to have its own shortcut methods for identifying ESSs, its own set of assumptions, and its own types of ESSs. The criteria are:

• Types of strategy set: The first criterion for classifying games involves the nature of the strategy set. The strategy set includes a list of alternative behaviors, physiologies, or anatomical structures that each individual in that role could adopt. These alternative strategies can either be discrete (e.g., produce a red tail or a green tail), or continuous (e.g., amplitude modulate a 1 kHz sine wave at some rate between 50 and 453 Hz). In either case, the strategy set may also permit mixtures of the alternative behaviors or structures (e.g., sing a 60 Hz song 20% of the time and a 130 Hz song the remaining 80%). As noted earlier, a strategy that consists of a single behavior or structure is called a pure strategy, whereas a stable mixture is often called a “mixed ESS.” Note that a mixed ESS can take one of two forms: a genetic polymorphism occurs when each member of the same role performs only one strategy, but different members of that role perform different strategies. For example, suppose 40% of male moths in a given species emit organic acid A as a pheromone, whereas the remaining 60% emit organic acid B. No male can emit both. Females, which represent an opposing role, perceive the male population as a 2:3 mixture of the two pheromones. The alternative form is a behavioral polymorphism. Here each individual performs a mixture of alternative strategies. Continuing our moth example, suppose all males emit pheromone A 40% of the time and pheromone B 60% of the time. Females still encounter male pheromones with a 2:3 ratio, but the mechanisms by which the mixture is achieved are different. Note that some authors restrict the term “mixture” to behavioral polymorphisms. While it usually does not matter to opponents whether a mixed ESS is generated through a genetic or behavioral polymorphism, evolutionary trajectories often differ for the two mechanisms and this affects the likelihood that a given role will ever achieve a stable mix (Bergstrom and Godfrey-Smith 1998). For example, inheritance patterns might allow behavioral polymorphisms but prohibit genetic ones. Strategy sets will thus differ in the degree to which one or the other form of mixed ESS is an option.

• Interacting player number: Evolutionary game theory initially focused on pairwise interactions. Even though there might be many animals in a population adopting multiple strategies, any given interaction was assumed to involve only two players at a time. Most of the examples we outline below make this assumption. However, there has been considerable effort in recent years to examine games in which three or more players interact at the same time and no payoffs are distributed until each individual has played its strategy. This provides an opportunity for players to cooperate, punish each other, or otherwise alter the strategies they might have adopted if only two-individual interactions were allowed. Trajectories for multi-person games can be quite complicated (Broom et al. 1997; Bach et al. 2006).

• Role symmetry: A third criterion for game classification is role symmetry. The opponent players in a symmetrical game have access to identical strategy sets, have equal chances of winning when playing a given strategy against another given strategy, and have identical payoffs when playing a given strategy against another given strategy. The players are thus entirely interchangeable; or put another way, there is really only one role. Two identically aged male baboons exchanging vocalizations over who gets a favored sleeping site are likely involved in a symmetrical game. In an asymmetrical game, there is more than one role, and each may have access to different alternative strategies, different probabilities of winning with a given strategy, different payoffs when they win with a given strategy, or some combination of these conditions. Asymmetric games are typical of interactions between male versus female, young versus old animals, or big versus small players. A vocalizing contest between a large adult male baboon and a small juvenile over a sleeping site is unlikely to be a symmetrical game as the probabilities of winning a fight are surely unequal, each individual is likely to have available strategies that the other cannot adopt, and the payoffs of winning may have different fitness consequences for each party. A single exchange of animal communication is always an asymmetrical game because sender and receiver do not have identical strategies. If there are multiple exchanges, and the parties alternate roles as sender and receiver, then the overall sequence can be symmetric.

• Contests versus scrambles: All evolutionary games are frequency dependent because the average payoff of a given strategy depends upon the current abundances of alternative strategies in the population. However, a second level of frequency dependence may or may not be present in an evolutionary game. If the payoffs for a single interaction depend only on which strategies each party adopted, we say the game is a contest. The payoffs of this single interaction are thus independent of the abundances of alternative strategies in the population. There is then no second level frequency dependence. However, if the payoffs in a single interaction vary depending upon the relative abundances of alternative strategies, the game is called a scramble (also called playing against the field). Some examples will help clarify this distinction.

  Consider first a population of animals that regularly hunt food that occurs only in patches or items so small that they are indivisible. Only one animal gets to eat such a find. When several animals find the same patch, each can adopt any of several fighting tactics to win control over it. Suppose that the probability that an animal using Tactic A will win when competing with an animal using Tactic B is fixed, as is the amount of food obtained if one wins. We can also include some fixed costs to each party depending on which tactics each adopts. Although there are a number of variables, and one can include some stochasticity in who wins, the expected payoff to each party for a particular combination of tactics is independent of how many other animals in the population have adopted those or alternative tactics. This is thus a contest. It is still a frequency dependent game in the first sense because the average payoff over of any given tactic over many interactions depends on the frequency with which an animal playing that tactic encounters opponents playing the same or alternative tactics. It is not however frequency dependent in the second sense defined earlier.

  Now consider male birds establishing nesting territories. There are two alternative strategies: settle in the thick woods or settle in the sparse woods. Thick woods provide more food and better cover per unit territory area and are thus preferred. However, as more and more males settle in the thick woods, they compress the territories of existing residents and make the average territory less and less suitable. Eventually, being the first settler in the sparse woods results in a territory that is as good or better than that obtained by being one more settler in the thick woods. The payoff of each strategy thus depends on how many males have already settled in each habitat. Because both the average payoff of adopting a strategy (frequency dependence in the first sense) and the payoffs of single interactions (frequency dependence in the second sense) vary with the relative abundances of the alternative strategies, this game is a scramble (Parker 1984; Krivan et al. 2008; Cressman and Krivan 2010).

• Temporal pattern: The fifth criterion has to do with the degree to which successive interactions should be treated as independent games or not. All animals face a sequence of decisions during their lives. Even during a single fight, courtship, or parent–offspring interaction, there may be several successive points at which each player must make a strategic choice. It is critical to know whether the outcome of an earlier choice of strategy does or does not affect the suitability of alternative strategies at a later decision point. If the earlier decision has no bearing on the later one, then we can treat each decision in the sequence as if it were an independent game. If the outcomes of early decisions do constrain later choices, then we must think of the entire sequence as the game, and each decision as a bout within that game. Such sequences are called dynamic games. An ESS is then an optimal choreography of successive strategy choices. This choreography will be fixed if the opponent’s choreography can be anticipated; alternatively, it may be conditional: e.g., if the opponent does A in the first bout, the best response in the second bout is then strategy 1; but if the opponent adopts B, then the focal animal should do strategy 2. In some dynamic games, one can identify an evolutionarily stable policy (Houston and McNamara, 1987; Mangel 1990), which is a guiding rule of thumb to find the optimal strategy at each point given current conditions. As examples, consider two male crickets that periodically try to take over the same mating burrow. If they have no signals that allow for individual recognition, and composition of the local population is sufficiently fluid, neither knows on successive occasions whether its opponent is a new individual or one it defeated (or was defeated by) on a previous occasion. Each contest may then be treated as an independent “single bout” game, and the optimal strategy for both crickets might be to threaten the other at the same level on each occasion. If, however, they can recognize and remember each other, then a prior winner would benefit from escalated aggression in subsequent interactions whereas a prior loser would do best to avoid confrontation. The latter series of interactions can only be analyzed as a dynamic game. An appropriate ESS policy might then adjust the aggressive levels of signals in successive bouts according to the fraction of prior wins (or losses) with a known opponent.

• Spatial pattern: Traditional evolutionary game theory assumed large populations and perfect mixing of animals adopting alternative strategies. As a result, the relative frequency of encountering an opponent playing a given strategy equaled the relative abundance of that frequency in the population. However, if offspring do not disperse far from parents and especially if there is an overlap of generations, or if animals adopting a given strategy preferentially settle near others with the same or complimentary strategies, the distributions of strategies can become quite heterogeneous spatially. The degree of spatial heterogeneity in the game in thus a sixth criterion. In spatially heterogeneous populations, the average payoff of adopting a given strategy can depend significantly on where the animal adopting it is located. We discuss spatial games in more detail in Chapter 15 of the text. A good introduction to this type of game model can be found in Nowak (2006).

• Combinations of criteria and finding ESSs: Games can thus be classified according to their values for each of these six criteria. All combinations of criterion values are possible: one game might be modeled as a discrete symmetric contest in which successive pairwise interactions are independent; another might be formatted as a dynamic and continuous asymmetric scramble involving 4 players per interaction. For each combination of criterion values, there are specific types of ESSs possible and specific methods for finding these ESSs. In addition, the recent advent of adaptive dynamics has made it clear that the number of possible trajectories and outcomes (above and beyond whether an ESS is present), also depend in complicated ways upon which combination of game properties are present. Many of these complications are well known from general treatments of dynamic analysis (Strogatz 1994). A few general guidelines may prove useful (Vincent and Brown 1988; Hofbauer and Sigmund 1998; Weibull 1998; Nowak 2006):

• Outcomes are more constrained and idiosyncratic for games with no overlap of generations (e.g., if difference equations are used for modeling) than if there is full and temporally random overlap (differential equations).
• The number of possible outcomes and trajectories in discrete games generally increases with the number of alternative strategies allowed. This is corresponds to a change in dimensionality in dynamic analysis.
• The number of possible outcomes and trajectories can increase if the number of players/interaction is increased (e.g., multiplayer games).
• The number of possible outcomes and trajectories are greater when the relevant fitness terms in modeled equations are nonlinear than when linear. For example, both linear and nonlinear systems can produce oscillating cycles in which the population mix of strategies keeps changing overtime but along a repeating trajectory. Stable oscillations in a linear system are neutral cycles around an equilibrium point (see the rock–paper–scissors game below). These are neutral in the sense that a nearby trajectory is not attracted to a given cycle, and the distance of the orbit around the equilibrium point is determined solely by initial conditions. Nonlinear systems can generate stable limit cycles in which sufficiently nearby trajectories are attracted and pulled into the limit cycle loop. The location of a limit cycle is set by the game dynamics and not necessarily by initial conditions. Most contests are linear games and most scrambles are nonlinear.

Finding ESSs in discrete single-bout symmetric contests

The games studied in this category are usually assumed to occur in very large (infinite) populations with complete mixing. We first outline how one might identify ESSs in these games generally, and then examine several important examples.

• General method for discrete symmetric contests: Discrete symmetric contests with no temporal or spatial patterns are the most easily analyzed games. This is because there is only one role, and the question is which strategy should be invoked when members of this role play the game against each other. The simplest case involves two player interactions where each player has access to the same two strategies. Because the game is symmetric, the two available strategies, S1 and S2, are the same for the two players. In such a 2 x 2 discrete symmetric game, there are four possible combinations of plays and thus four possible payoffs to each player. ESSs are most easily found using a normal form matrix as shown in Figure 1. One player is arbitrarily assigned the focal status, and the matrix then lists the payoffs it would receive given each strategy it can play and each strategy its opponent can play. For a symmetric game, it does not matter which player is selected as the focal individual.

  Figure 1: Payoff matrix for 2 x 2 discrete symmetric contest. Because the game is symmetric, the two players are interchangeable and we only need to list one payoff for each pair of strategies played. The convention is to list the playoffs to the player on the left. Payoffs are given in the same currency for all cells in the matrix and denoted here by PO_ij. The first subscript (i) is the strategy played by the focal player and the second (j) is that of its opponent.

  To find the ESSs in this game, we can use the “dot method.” First, we look at the left column in this matrix. In this column, the opponent plays S1. What is the best response of the focal player to this move? Assuming that the payoffs (denoted PO_ij where i is the focal player’s strategy and j is the opponent’s) are in units of fitness and that natural selection will tend to maximize fitness, the best response by the focal role is S1 if PO11 > PO21, and S2 if PO₁₁ < PO₂₁. We put a dot in the larger cell in this column. If the two payoffs are equal (e.g., there is a tie), then it does not matter which strategy is used as a response and we put dots in neither cell. We then look at the second column in which the opponent will play S2. In this case, we compare PO₁₂ to PO₂₂ and place a dot in that cell with the larger value. If there are no ties in either column, the outcome has to be one of the four cases shown in Figure 2:

  

  Figure 2: Possible ESSs in 2 x 2 discrete symmetric contest. For each column in the payoff matrix, a dot is placed in that cell conferring the best payoff to the focal player. There are four possible outcomes. Case I: S1 is the best response regardless of what the opponent plays. S1 is thus a pure ESS. Case II: S2 is the best response regardless of the opponent’s play. It is then a pure ESS. Case III: The best response is the strategy opposite to that played by the opponent. If mixtures are allowed, the ESS is a fixed mixture of S1 and S2. Case IV: The best response is to match the play by the opponent. Which strategy is the ESS depends upon the initial frequencies of the two strategies in the population.

  In Case I, the best response is always S1, regardless of which strategy is employed by the opponent. S1 is then the ESS and because it consists of a single strategy (as opposed to a mixture of strategies), it is a pure ESS. In Case II, the best response is always S2. We can then call S2 the pure ESS. If there is a tie in one of the two columns, then that column exerts no selective force on the choice of strategy by the focal player. The row with a dot in the untied column is then the ESS. If both columns are tied, then there are no selective forces on strategies and hence no ESSs.

  Case III is a bit more interesting. In this case, the best response is the strategy not used by the opponent. Imagine a population in which all players only performed S1. A mutant animal which hit on S2 would have higher fitness than the majority playing S1 and S2 would increase in frequency with successive generations. Similarly, if we started with a population in which all played S2, a mutant performing S1 would be able to invade and this strategy would become increasingly common over time. If S2 is better when S1 is common, but S1 is better when S2 is common, it seems obvious that there ought to be some intermediate mixture of S1 and S2 at which neither has an advantage over the other. Any time the population drifts away from the optimal mixture, the rarer strategy will have an advantage and will increase to bring the population back to the ESS. The system is thus convergence stable even in the face of some drift in frequencies. Note however that a mixed ESS cannot be a fitness maximum since playing either pure strategy or the mixed ESS, once the population has reached the ESS, will yield the same payoff.

  What would the ESS mixture be? Suppose we let f be the fraction of the population exhibiting S1 at any time. A focal player adopting S1 will encounter opponents playing S1 a fraction f of the time and on each occasion will receive a payoff of PO₁₁. It will also encounter opponents playing S2 a fraction (1–f) of the time and on each occasion will receive a payoff equal to PO₁₂. Recalling the methods for calculating average payoffs detailed in Web Topic 8.1, we can compute the average payoff to a focal player always playing strategy 1 as

  

  Similarly, if the focal player were always to play S2, its average payoff would be

  

  We have argued that there must be some value of f, denoted by f_e, at which equilibrium occurs and thus it does not matter which strategy you play. This means that

  

  when f = f_e. Setting the above equations equal to each other and solving for f_e, we get that

  

  In words, the fraction of the time we should see S1 at equilibrium is equal to the absolute difference between the values in the right hand column of the matrix (the one occurring with frequency [1–f]), divided by the sum of the absolute differences between the values in each column. Clearly, this must be some fraction between 0 and 1.

  As we noted earlier, a mixed ESS might be achieved by a genetic polymorphism, in which f_e of the players always play S1 and (1– f_e) of the players always play S2, or by a behavioral polymorphism in which each player performs S1 a fraction f_e of the time and performs S2 the remaining (1 – f_e) of the time. As far as the analysis is concerned, the two solutions are equivalent. In real life, not all genetic systems will allow the establishment of a genetic polymorphism at any given frequency f_e. Genetic impediments are less likely for behavioral polymorphisms.

  Case IV is the reverse of Case III: the optimal choice is to match your opponent’s choice of strategy. If we imagine an initial population all performing S1, we can see that S2 is always at a disadvantage and thus cannot invade. If we begin with a population all playing S2, S1 is now the less favored option. Again, it seems likely that there ought to be some intermediate equilibrium at which payoffs of playing either are equal. In fact, the calculation of f_e for Case IV by setting

  

  gives exactly the same result as for Case III. However, in this case the equilibrium point is not stable. Suppose one begins with a population at this equilibrium mixture. Barring drift, it would pay no player to shift strategies and thus it is a Nash equilibrium. However, if the frequency of players adopting S1 drifts upwards so that f > f_e, S1 is now the better strategy and subsequent evolution will drive the system towards 100% of the animals using S1 (e.g., to f = 1). If f had drifted lower so that f < f_e, S2 would be favored and S1 would disappear (e.g., f = 0).

  Case IV thus has two ESSs: either pure S1 or pure S2. Which is observed in the short term depends upon whether the initial frequency of S1 is greater or less than f_e. Over the long term, most finite populations will experience enough drift for moderate numbers of the initially rarer strategy to accumulate. Although one strategy may be most common and thus the ESS at the start, drift may eventually push f past f_e causing the system to flip to the other ESS. The ESS most likely to be found over the long term is the strategy which is least common at the equilibrium point. The logic is as follows. Suppose f_e is less than 0.5: that is, the frequency of S1 at the equilibrium is much less than that of S2 (e.g., f_e << [ 1– f_e]). If we begin with a population at the S2 ESS (f ≈ 0), S1 players will appear only as rare mutants (genetic heritability) or innovators (cultural heritability). If f_e is a small enough value, only a few such mutants are needed before f exceeds f_e. Once this occurs, S1 becomes the favored strategy, and the system will move to an S1 ESS. By the same token, if we begin with a S1 ESS (f ≈ 1), it will take a very large number of S2 mutants, since f_e is small, before f can drift below f_e and cause a switch to S2. The strategy which is least common at the equilibrium is thus the one which is most likely to invade by chance if the other is the ESS, and the one least likely to suffer invasion when it is the ESS. As we saw above, the value of f_e is determined entirely by the relative payoffs in the matrix. A simpler shortcut to find the more likely ESS is to compare the absolute values of the differences between payoffs in each column of the payoff matrix. The column with the greatest absolute difference (called the risk dominant strategy), is the one that is most likely to arise over evolutionary time.

  Note that a 2 x 2 discrete single-bout symmetric contest with no ties always has at least one ESS (as long as mixed ESSs are allowed). As discussed later, this may not be true for discrete symmetric contests with more than two strategies.

• Well-known examples of 2 x 2 discrete single-bout symmetric contests: Just as there are three general outcomes in the 2 x 2 games reviewed above (a single pure ESS, a mixed stable ESS, or two alternative pure ESSs), there are three corresponding games that have had significant impact in evolutionary biology and behavioral ecology. Each of these has been given various names and incarnations, but the predictions from each have proved remarkably robust, even when scaled up to more complicated versions.
  • Take games: Take games have two basic strategies: passive minds its own business, and cheat exploits others. As an example, consider foraging terns. Passive individuals ignore others and have an average catch of P fish per time interval. Cheats also fish on their own but when they spot another tern with a captured fish, they dive at it and steal the fish. When a passive and a cheat hunt in the same area, the passive tern loses B fish to the nearby cheat. Its average payoff is then P-B. The cheat gains B fish from the passive, but at a loss of -C fish that it might have caught had it kept fishing on its own. Its payoff is then P+B-C. When two cheats meet, half the time one gets to steal B fish from the other at a cost -C, and the other half of the time, it has B fish stolen from it. The average payoff is then 0.5(P+B-C) + 0.5(P-B) = P-0.5C. The payoff matrix for this evolutionary game is then:

    Figure 3: Payoff matrix for a take game involving foraging terns. See text for justification of payoff values.

    If B > C, then P+B-C > P, and cheat is the better strategy in a population of passives (left hand column). We can thus put a dot in the lower left cell of the matrix. Cheat is also the best response in a population of cheats (right hand column) if P-0.5C > P-B. This will be the case if B > 0.5C. If B > C (left hand column condition), then it is also true that B > 0.5C (right hand condition). In short, if B > C, then cheat is a pure ESS (Case II).
    
    Note that at the ESS, everyone does worse (e.g., P-0.5C) than they would have done if they had all remained passives (e.g., P). A payoff which is higher if everyone avoids cheating is called a pareto optimum. However, there is always a temptation to cheat, and once some cheating starts, eventually all players have to become cheaters just to keep up. Some factor outside of the game would be required to keep the players at the pareto optimum.
    
    Take games appear in a wide variety of incarnations, but they invariably give the same result: passive independence is vulnerable to invasion by selfish individuals with the result that everyone ends up worse in the end. Arms races are a classic example of a take game. A second famous example is the prisoner’s dilemma in which two prisoners would do best if neither provides evidence on the other, but the temptation to get more lenient treatment causes one to defect, and the ESS is then for both to defect with much worse consequences than if they had both kept silent. We discuss this game further in Web Topic 13.1. Herd animals that try to put themselves closer to the herd center to reduce their own predator vulnerability (the selfish herd) are playing a take game (Hamilton 1971). In evolutionary ecology, the red queen hypothesis (Hamilton 1980; Bell 1982; Salathe et al. 2008) can also be viewed as a take game. Theft and unregulated capitalism can both lead to take games in human societies: one purpose of laws and regulations is to try to keep the population at a pareto optimum. The contexts vary: the outcome is routinely the same.
    
  • Give games: The opposite case is called a give game (also called a coordination game in the economic literature). Instead of food acquired, we here use a general currency, fitness units, for payoffs. There are two strategies: passive players, as before, mind their own business, whereas donors can give up to B fitness units to half of the other players that they encounter at a cost of –C fitness units to themselves. When donors encounter other donors, half the time they gain B fitness units from the other player’s donation, and half the time they lose -C fitness units by donating to the other player. As long as B > C, exchanges between donors can be favored by selection. However, there is no way donor mutants could invade a population of passives as their fitness would be P-0.5C, whereas the average fitness of passives when the mutants were rare would be P. Even if donors were common, being a passive would have a higher payoff (P+0.5B) than being a donor (P+0.5B-0.5C). Passive would remain a pure ESS. One way out of this dilemma is for donors to be selective to whom they donate. This might involve individual recognition and memory of prior interactions, kin recognition, or some other factor that allowed them to identify who was and was not a donor. If they then reduced their rate or amount of donation to passives, so that the average donation to a passive was b < B and the cost to the donor was k < C, could being a donor then be an ESS? And if so, how selective would they have to be? The relevant normal form matrix is shown in Figure 4:
    

    Figure 4: Payoff matrix for a give game involving selective donation. See text for explanation of payoff values.

    It is clear that as long as there is some residual cost to donating fitness units to passives, the passive strategy is the best response in a population of passives (P > P – k). We thus put a dot in the top cell in the left hand column. If (B-C)> 2b, then donor is the best response in a population of donors. Thus if donors can be sufficiently selective, either passive or donor can be a pure ESS depending upon initial conditions. Donor can be the more likely ESS if (B-C)> 2(b+k).
    
    This points up a general outcome for give games: charitable cooperation is at best a Case IV situation; there are invariably two possible ESSs, one of which is not to cooperate, and no guarantees that the charitable cooperation outcome will be the one seen in nature. Put another way, it really does not pay to be a donor unless there are already enough other donors around. This raises the question of whether charitable cooperation could ever invade a population of passives. A variety of mechanisms have been suggested including initial cooperation due to kin selection or heterogeneous settlement patterns to get past this hurdle. The bottom line is that selfish arms races are the common outcome of take games, but only under certain circumstances can give games end up at a cooperative ESS. We discuss give games in more detail in Web Topic 13.1.
  • Stable mixture games: Stable mixtures are the expected outcome whenever the best response is the opposite of what one’s opponent plays. Perhaps the best known example is the hawk and dove game (Maynard Smith 1982). The biological context is a conflict between two individuals over some indivisible commodity such as a single food item, a mate, or a roost. Winning the commodity increases the fitness of the winner. The naive expectation might be that the two animals should always fight over the commodity. The observation in nature is that animals often use relatively low risk display signals to decide who should get the commodity; one animal then leaves without a fight. It is not immediately clear why the two animals do not always fight over the commodity. Hawk versus dove is a simplified game invoked to explain these observations. It assumes that all contestants are equal (making the game symmetric), and allows for only two discrete strategies: fight (hawk) and display peaceably (dove). Only two players are allowed to encounter the same commodity at a time and payoffs of a single interaction are independent of how many players are hawks or doves. The game is thus a contest. Players have no memory and may never meet again; thus this is not a dynamic game.
    
    We again use fitness units as the relevant payoff currency. The commodity is worth V > 0 fitness units and the fitness cost of losing an escalated fight is –D. Two hawks always engage in an escalated fight during which one wins and the other loses. Since they have equal chances of winning or losing, the average payoff of such a fight is 0.5(V-D). If a hawk and a dove jointly encounter the commodity, the dove flees with payoff 0 and the hawk gets it with payoff V. If two doves meet, they coo at each other with negligible costs until one (randomly determined) gives up. The average payoff for two doves is thus 0.5V + 0.5(0) = 0.5V. The payoff matrix is shown in Figure 5.
    

    Figure 5: Payoff matrix for the simplest version of the Hawk–Dove game. See text for payoff explanations.

    The dot in the right hand column must go in the upper right cell since V > 0.5V for any positive V. The dot in the left hand column would go in the upper left cell if V > D. Hawk would then be a pure ESS (not shown in figure) and the animals would always fight over the commodity. But if the costs of injurious fighting exceed the value of the commodity, then V < D and the dot would go in the lower left cell. This is then a Case III game and one would expect to find either a genetic or behavioral polymorphism in which hawks represented a fraction:
    One can make this game much more complicated (see several chapters in the text), but the basic message is that if the costs of aggressive actions are high enough, it may pay both parties to resolve the dispute by some form of display at leas some of the time.

• Discrete single-bout symmetric contests with three strategies: While all of the two-strategy games discussed in the prior section have at least one ESS, this is not necessarily the case for otherwise similar games with three or more strategies. One very illustrative example is the rock–paper–scissors game (Maynard Smith 1982; Weissing 1991). This game has three strategies with the properties that rock beats scissors, scissors beats paper, and paper beats rock. The classical game has the payoff matrix shown in Figure 6A:

  Figure 6: Payoff matrices for rock–paper–scissors games. (A) Classical payoffs normalized so that each combination of plays results in a gain or loss of the same amount. (B) Payoffs for generalized version of game in which different payoffs can occur for different combinations of plays. Note that games with rewards (payoffs >0) or penalties (payoffs < 0) for ties (cells along the main diagonal) can be renormalized in either version without changing the dynamics or equilibria of the game by adjusting column values so that the main diagonal only has zeros.

  Visual inspection of the matrix in Figure 6A suggests that no pure strategy can be an ESS in this game: each strategy can be invaded by one of the others. However, one might imagine that there could be some sort of equilibrium mixture. In fact there is, but the dynamics of this game may or may not lead a population to that equilibrium. The equilibrium for the Figure 6A version occurs when each strategy constitutes 1/3 of the population; the equilibrium might occur at some other mixture for the generalized game of Figure 6B. It can be shown that the dynamics of these games can be predicted by calculating the sign of the determinant of their payoff matrices (Weissing 1991; Hofbauer and Sigmund 1998; Nowak 2006). There are three possible cases that are best visualized using the simplex of the game (Figure 7). Since there are three discrete strategies, this will be an equilateral triangle, and since no pure strategy can be an ESS, any interesting trajectories will in the triangle’s interior.

  

  Figure 7: Evolutionary dynamics of rock–paper–scissors game plotted on the relevant strategy simplex. Red dot shows equilibrium point. (A) If the determinant of the payoff matrix is zero, the population will follow a neutral (fixed) limit cycle around the single equilibrium point. The distance of the cycle from the equilibrium will depend on the starting point of the population. Two possible trajectories are shown. (B) If the determinant of the payoff matrix is positive, the population will follow a decreasing spiral that asymptotically approaches the equilibrium point. (C) If the determinant is negative, the dynamics will follow an increasing spiral that asymptotically approaches the edges of the simplex.

  As shown in Figure 7, there are three possible outcomes. The classical rock–paper–scissors game of Figure 6A has a determinant equal to zero: the population will cycle around the equilibrium point without getting either closer or further away on successive loops (Figure 7A). The starting mixture of strategies in the population determines the location of the loop within the simplex. If the determinant is positive, the population mixture also cycles around the equilibrium, but on each pass it gets closer to the equilibrium. It thus approaches the equilibrium asymptotically (Figure 7B). If the determinant is negative, the population is also a spiral but in this case, each loop takes the population further from the equilibrium. What happens once it is asymptotically close to the simplex boundaries depends on the values of the matrix and the degree to which evolution in this population is subject to drift. It is instructive to note that replicator and standard ESS analyses will often give similar predictions for this game as long as generations are overlapping (e.g., allowing differential equations of change) and payoffs are not too heterogeneous. However, once these conditions are not met, the replicator dynamics and ESS solutions can make rather different predictions: what may be stable for one generational schedules may not be in the other (Weissing 1991).

  The point of this example is that once there are three or more strategies in an evolutionary game, evolutionary trajectories can become quite complicated. There may be NO ESS and any equilibria may only function as reference points for cyclic changes in the current mixture of strategies. Such rock–paper–scissors-like cycles have actually been found in microbes, sessile animals, and the mating and communication strategies of some lizards (Alonzo and Sinervo 2001; Frean and Abraham 2001; Sinervo et al. 2006; Sinervo et al. 2007).

• Discrete symmetric contests with more than three strategies: As noted earlier, discrete games with more than three strategies can exhibit additional trajectories and outcomes not seen in lower dimensioned games. It is impossible in most cases to identify ESSs in this type of game with more than three strategies, and even in some with three strategies, using dot methods or simple inspection. Occasionally, one can obtain insights by examining sub-games within the larger matrix. However, usually, one must resort to finding equilibrium points that meet Lyapunov conditions, or attempt to linearize the curvature of the adaptive surfaces near equilibrium points and use higher order derivatives (e.g., a Jacobian matrix) to assess equilibrium stability (both at the point and locally) (Hofbauer and Sigmund 1998; Cresssman 2003; Leimar 2009).

Finding ESSs in discrete single-bout asymmetric contests

• General methods for asymmetric discrete contests: We now consider discrete contests in which there are two separate roles. The appropriate payoff matrix will need to show two payoffs for each combination of strategies (one for each player). The convention (in behavioral ecology) is to divide the matrix cell for each pair of strategies diagonally in half and to enter the payoff to the player on the left in the lower left corner of the cell, and that to the player at the top in the upper right hand corner (Figure 8).

  Figure 8: Payoff matrix for discrete 2x 2 asymmetric contest. In this example, there are two players who represent different roles (A and B) and each role has access to two alternative strategies. Because the strategies available to the roles in an asymmetric game are usually different, we assign strategies one and two to role A and strategies three and four to role B. Any combination of strategies by the two players may result in different payoffs for the two players. As before, we use subscripts to indicate the strategy used by player one followed by the strategy used by player two. In parentheses, we indicate to whom that particular payoff is given.

  A 2 x 2 asymmetric discrete contest may have no ESS, one ESS, or several ESSs. It will not have mixed strategies similar to those seen for symmetric contests if players are fixed in a given role and cannot, by definition, perform one of their own strategies part of the time and one of the strategies of another role the rest of the time. If animals are not fixed in a given role, as might occur if they make mistakes about their relative roles, then mixed asymmetric ESSs are possible.

  To identify the pure ESSs (if any) in a 2 x 2 discrete asymmetric contest, we use the “arrow” method. This indicates the best responses by placing arrows parallel to each of the four sides of the payoff matrix. For example, we might first consider what the player in role A of Figure 8 should do if its role B opponent plays strategy three. If PO₁₃ (A) > PO₂₃ (A), then the role A player should adopt strategy one over strategy two. We would then place an arrow parallel to the left side of the payoff matrix pointing up. If PO₁₃ (A) < PO₂₃ (A), then we make that arrow point downwards. We then consider what the role A player should do when the role B player opts for strategy 4. If PO₁₄ (A) > PO₂₄ (A), then we place an arrow parallel to the right side of the matrix pointing up; if the converse is true, then we point the arrow down. We next consider the optimal strategies for the role B player. If player A elects to use strategy one, what should the role B player do? If PO₁₃ (B) > PO₁₄ (B), then the role B player should adopt strategy three. We then place an arrow parallel to the top of the matrix pointing to the left. If the converse were true, we would make that arrow point to the right. Finally, we identify the best response for the role B player when its opponent adopts strategy two. This results in an arrow parallel to the bottom of the matrix pointing either to the right or to the left.

  Any point (marked with an asterisk) at which two arrowheads meet is an ESS for that game. For a simple 2 x 2 discrete asymmetric contest, there are 16 possible arrangements of the arrows and 3 classes of outcomes (Figure 9):

  

  Figure 9: The 16 possible arrow arrangements and 3 classes of outcomes for 2 x 2 asymmetric contests. See text for details.

  Of the 16 possible arrow arrangements, two will result in no ESSs. The four arrows either circle around the matrix in the clockwise direction or in the counterclockwise direction (as shown here). The outcome is usually a stable population cycle. For each of the four corners, there are three ways to arrange the other arrows so that they do not meet. This generates 12 ways in which the game can result in a single ESS. Finally, there are only two ways that one can get two ESSs since these must be located diagonally across the matrix from each other. As with Case IV for symmetric 2 x 2 contests, which ESS one finds usually depends on initial conditions.

• Asymmetric Hawk and Dove: An example will help demonstrate this approach. Consider a version of Hawk–Dove game in which the two interactants have different probabilities of winning an escalated fight over a contested commodity (Hammerstein 1981). As in the symmetric example above, let the value of the commodity be V fitness units, and the injury cost of losing an escalated fight be –D. The only difference from the earlier version is that the probabilities of winning an escalated fight are no longer 0.5 for each party: instead, the dominant animal has a probability of P_D >0.5 of winning such a fight, whereas the subordinate has a chance of P_s < 0.5 of winning. We shall assume that if the two parties simply display at each other, they have an equal chance of giving up first. The resulting payoff matrix is shown in Figure 10:

  Figure 10: Payoff matrix for asymmetric Hawk–Dove game. The only difference between this and prior symmetric version is an unequal probability of the two contestants winning an escalated fight.

  As long as V > 0, we can draw two arrows immediately: the arrow on the right hand side should point upwards (since V > 0.5V) and that below the matrix should point to the left (for the same reason). The questions are then which directions the top and left hand arrows will point. If P_SV–P_DD > 0, the top arrow will point to the left; if P_SV–P_DD < 0, it will point to the right. If P_DV–P_SD > 0, the left hand arrow will point upwards; if P_DV–P_SD < 0 it will point downwards. There are three possible cases which are shown in Figure 11:

  

  Figure 11: Three possible outcomes of asymmetric Hawk–Dove game. ESSs are shown as asterisks and relevant arrows are drawn. Conditions guaranteeing this distribution of arrows are provided in similar format below each example.

  Since the three conditions shown in Figure 11 all scale according to the same parameters, another way to present the outcomes is shown in Figure 12:

  

  Figure 12: Relationships of outcomes in asymmetric Hawk–Dove game to asymmetry in winning escalated fights (P_D) and relative cost of losing an escalated fight (D/[V+D]).

  Figure 12 presents quite intuitive results. At low relative costs of losing an escalated fight (small D/[V+D]) and small asymmetries in the chance of winning (small P_D), the ESS is for both the dominant and the subordinate to play Hawk and escalate (Case I). The subordinate will lose more often than the dominant, but the chances of winning are still good and the cost of losing negligible. However, as the probability P_D that the dominant wins and/or the costs of losing a fight D/(V+D) increase, the ESS shifts to one in which the dominant routinely plays Hawk and the subordinate plays Dove (Case II). If the cost of losing a fight is large and/or the value of the commodity small, either of two ESSs can evolve: one in which the dominant plays Hawk and the subordinate retreats, or one in which the subordinate plays Hawk and the dominant retreats. In this context, it is not worth fighting over the commodity and even a somewhat non-intuitive convention, such as the subordinate challenging the dominant and the latter retreating, resolves the issue quickly and simply.

Finding ESSs in continuous contests and scrambles

Any of the examples in the previous section can be made into continuous games by allowing players to vary their strategies in graded ways. Thus Hawk might grade into Dove by varying the amount of energy put into an attack. The strategy set is then the amount of energy invested in escalations. Similarly, a give or take game can be made continuous by varying the amount of resource given or taken per interaction. The question is then what is the ESS amount to give or take. Even the behavioral polymorphism ESSs of discrete games are continuous strategies: what is the optimal fraction of time that a player should spend on each strategy?

• General method: Games with continuously variable strategy sets invariably have continuous functions relating fitness to an actor’s strategy and the current population mix. The obvious (and traditional) way to find ESSs in continuous games is to compute the first derivative of this function and set it equal to zero (to identify any equilibria), and then ensure that the second derivatives of the function at each equilibrium are negative (implying a fitness maximum) (Maynard Smith 1982; Parker 1984). These will then be ESS values of the strategies. As mentioned earlier, more recent approaches also require examination of the behavior of the function at points in the vicinity of the equilibrium to determine whether the latter is convergence stable or not. The use of Lyapunov functions or linearization and subsequent eigen-analysis are now widely used for this task (Hofbauer and Sigmund 1998; Weibull 1998; Cresssman 2003). Note that just because a strategy set and the corresponding fitness functions are continuous does not mean that the ESS cannot be discrete: as we shall see in the example below and in certain other contexts, continuous games can have separating, pooling, or semi-pooling equilibria depending on the game dynamics. In the latter two cases, the equilibrium calls for a segmentation of the continuous strategy sets, and even players with different attributes (roles) can be assigned to the same segment (Parker 1982; Bergstrom and Lachmann 1998; Lachmann and Bergstrom 1998; Bergstrom et al. 2002).

  Scramble games almost always have continuous fitness functions even if the alternative strategies are discrete (e.g., settling in habitat A versus habitat B). Where these continuous functions for a scramble game can be defined, the same utilization of derivatives of those functions can be undertaken to find ESSs and assess convergence stability. It should be noted however that while fitness is usually a linear function of strategy frequencies for contests, it will be nonlinear for scrambles. This means that the dynamics and possible outcomes for scrambles can be much more complicated than those of a similarly dimensioned continuous contest.

• Two classic continuous contests: In the analysis of the Hawk and Dove game, we argued that two Doves resolved their conflict over a commodity by displaying at each other until one gave up. We did not discuss why or when either should give up the contest. One of the classical solutions to this problem is the war of attrition. It comes in two forms: symmetric and asymmetric. Since these games are cited in Chapter 11 of the text, we briefly review them here:
  • Symmetric war of attrition (Mayard Smith 1974; Parker 1974; Bishop and Cannings 1978; Bishop et al. 1978): Consider two opponents each of which competes for some commodity V by selecting some amount of display time t from a continuous distribution. Neither knows before the confrontation what display time has been chose by their opponent, and no party is allowed to gain additional information while interacting. During the confrontation, the opponent that selected the longer display time wins the commodity (hence the name “war of attrition”). If both pick the same t, then they settle the interaction by chance. Once the confrontation is over, each party will have paid some cost of display: this will be the same for both parties since this is a symmetric game and they each displayed for the time selected by the loser. Let this cost be -kt where k is the rate of loss per unit time displaying in the same currency as the commodity V. In a population in which all players use the same t, the average payoff would be 0.5V –kt (since all confrontations are ties).

  It should be obvious that there is no pure ESS in this game: a population in which all animals used a given display time could be invaded by a mutant picking a slightly longer time. Eventually, once all animals expended kt > 0.5V, a mutant that selected t = 0 could invade. No pure strategy is uninvadable. On the other hand, there is a mixed ESS in this system in which each player selects a display time t with probability p(t):

  

  This is a negative exponential distribution: most individuals will select t =0, but other values are possible with increasingly longer values of t being increasingly rare (Figure 13A).

  

  Figure 13: Mixed ESSs for wars of attrition games. (A) Symmetric war of attrition ESS is a random draw of a display time (t) from a negative exponential distribution (p(t)). All players draw from the same distribution but will usually draw different times. Individual with longest time wins. Duration of contest is set by individual with shortest time. Note that individuals using a given t all the time will get the same payoff as individuals drawing randomly from the distribution as long as the overall distribution is maintained. (B) Asymmetric war of attrition ESS consists of two negative exponentials: one for those who think their V/k ratio is higher than that of the opponent (to the right of S), and one for those who think their V/k ratio is less than that of the opponent. Each individual should decide on which side of S it sits and draw a display time randomly from that distribution. See text for determination of S.

  • Asymmetric war of attrition (Parker and Rubenstein 1981; Hammerstein and Parker 1982): Now consider the same game as above except that either the value of the commodity, V, or the costs of display, k, differ between the two parties. As long as one or both parties can at least partially assess their own and their opponent’s values of the ratio, V/k (e.g., based on body size, age, prior experience, etc.), the ESS is to draw a random display time from one of two distributions (Figure 13B). If a contestant thinks it has a lower V/k, it selects a time from the distribution of t values < S; if it thinks it has a relatively high V/k ratio, it should draw a random time from the distribution of t values > S. Note that this ESS only exists if players have imperfect information: were a player to know for sure who would win a protracted display contest, there is no reason to display at all. Note also that it is possible that both players might draw display times from the same distribution. The ESS value of S will vary with the relative values of V and k for the two parties, and with the probability, Q, that each animal accurately assesses it and its opponent’s relative V/k ratios: specifically, S increases with an animal’s own V, decreases with its k, and decreases with its ability to accurately assess the situation Q. Put simply, if animals make few mistakes in judging their V/k relative to that of the opponent, S will be a small number, and individuals will only draw from the longer t distribution if they are fairly sure they will win. If mistakes are common, S will be a larger number and individuals are more likely to draw from the lower t distribution. As assessment errors and S increase, the likely variation in contest times is also predicted to increase. This latter prediction is often met in real situations: where displaying contestants are similar in condition and endurance, one often sees much more variability in contest durations than if they are highly asymmetrical.

Some additional considerations

Most of the evolutionary games that arise in animal communication contexts can be analyzed with the methods outlined above, although implementation can become algebraically complex. However, there are several communication contexts in which the classical expectations are constrained or additional factors relevant and this can change the analysis. We outline several of these contexts below:

• Sequential games: A number of the evolutionary games that arise in animal communication are sequential games: instead of each party deciding ahead of time which strategy to play and then playing them simultaneously, the parties make their plays successively. Optimal strategy selection for all but the first play is thus conditional on what the other player just did. The sequential assessment game for conflicts discussed in Chapter 11 is a good example. If there are only two stages or bouts, ESSs may be identifiable by examining normal form matrices. However, even in this case, extensive form analyses may better clarify assumptions and game logic. For example, Hurd (1995) used extensive form analysis of sender/receiver interactions to show how sender costs might stabilize communication when the two parties had disparate interests. Hurd and Enquist (2005) then extended this approach to classify communication games. For complex sequences, some form of dynamic programming (also called backward induction) is commonly employed to identify ESSs (Mangel and Clark 1988; Houston and McNamara 1999). Even more complicated sequences may require simulation methods (Hamblin and Hurd 2007). Note that where the sequence is stretched out over time, for example, in a life history game, fitness changes at each bout may need to be combined multiplicatively instead of additively (Seger and Brockmann 1987; Getty 2006). Cressman (2003) provides a general review of extensive form game analysis and shows how evolutionary dynamics can be combined with this approach. Gintis (2000) examines a wide variety of evolutionary and economic games in extensive form. One construct that has proved very useful in analyzing sequential games is the notion of state: this might be a running measure of energy supplies, body integrity, information (e.g., the sequential assessment game), or some other variable that changes cumulatively during successive bouts. For a detailed explanation of how state can be incorporated into sequential evolutionary games, see Houston and McNamara (1999).

• Finite populations: As discussed earlier, traditional analyses of evolutionary games assume infinitely sized populations. This minimizes the stochastic effects of drift. Once focal populations are finite, drift cannot be ignored and becomes increasingly important as the population size gets smaller. A strategy that would always be a pure ESS in an infinite population can in fact be the one that goes extinct first in a small finite one. In addition, a small number of mutants in a small population can significantly change strategy frequencies and thus change the adaptive landscape for evolutionary games in that population. The assumptions used to define an ESS for infinite populations may thus no longer hold and require different definitions for stability (Fogel et al. 1998; Traulsen et al. 2005; Fudenberg et al. 2006; Traulsen et al. 2006a; Traulsen et al. 2006b; Traulsen et al. 2007; Zhou et al. 2010).
  

  An excellent introduction to evolutionary games in finite populations is provided by Nowak (2006). Whatever population size is being examined, he assumes that it is stable so that each birth results in a corresponding death and vice versa (called a Moran process). Game payoffs that have only a slight effect on a player’s fitness exert only weak selection, whereas those with a major effect exert strong selection. The possible outcomes of evolutionary games in a finite population, even for a simple 2 x 2 discrete symmetric contest, can be quite different from the classical case depending on both population size and the level of selection. For example, Nowak cites a 2 x 2 matrix in which strategy A is clearly the pure ESS for infinite populations. However, B can be an ESS in very small populations, both A and B can be ESSs in intermediately sized populations, and only for large populations does the game converge on the infinite case with A as the only ESS. Another example involves any Case IV game in which there are two ESSs: for infinite populations, the risk dominant strategy is the one expected to be seen most commonly. However, in large but finite populations under weak selection, a strategy can only take over the population if its frequency at f_e < 1/3 (Nowak et al. 2004; Ohtsuki et al. 2007a). This is a more stringent condition than being risk dominant (where its strategy at f_e only needs to be < 1/2). Altrock and Traulsen (2009) find similar differences in the dynamics of the rock–paper–scissors game when played in finite instead of infinite populations. Not surprisingly given these patterns, both the arms races of take games and the potential cooperation of give games can exhibit quite different outcomes when played in finite populations (Nowak et al. 2004).

• Heterogeneous populations: As with finite populations, the predicted ESSs of evolutionary games in heterogeneous populations may differ significantly from those of homogeneous (well-mixed) populations. The primary reason is that the average payoff for a given player is not based on the global frequencies of the alternative strategies, but instead on some biased subset of these. There are two (at least) ways that this might happen. First, animals might only have interactions with spatial neighbors. If a population is viscous, such that offspring do not settle far from parents, and all interactions are local, a focal player is unlikely to encounter strategies at rates equal to at their global frequencies. Alternatively, individuals may have the mobility or linkages needed for exposure to global strategy frequencies, but they actively limit their interactions to other players who adopt particular strategies (Pacheco et al. 2006).

  There has been considerable research done on spatially heterogeneous evolutionary games (Nowak and May 1992; Taylor et al. 2004; Ohtsuki and Nowak 2006; Santos et al. 2006; Taylor and Nowak 2006; Grafen 2007; Lehmann et al. 2007; Ohtsuki and Nowak 2007; Ohtsuki et al. 2007b; Grafen and Archetti 2008; Tarnita et al. 2009a; Tarnita et al. 2009b; Nowak et al. 2010). Heterogeneous populations are best modeled as networks, a topic we discuss in some detail in Chapter 15. Note that most spatial game analyses also assume that the population is finite. Nowak (2006), again, provides a good introduction to spatial evolutionary games in finite populations. One interesting outcome of recent work is that a parameter that depends only on the network structure can be used to predict the outcome of any 2 x 2 discrete symmetric contest even if the population is heterogeneous and finite (Tarnita et al. 2009b). This means that the same payoff matrix might result in quite different ESSs depending upon the population’s network structure.

• Genetic constraints: Traditional evolutionary game models assume haploid genetics and only rare mutations. What happens to evolutionary game predictions in populations of typical diploids with realistic rates of recombination and mutation? For pure ESSs, the outcomes are usually unchanged for diploid polygenic systems, although the dynamics may be more complicated. Mixed ESSs, however, pose a more complicated problem. For example, the symmetric war of attrition mixed ESS is not attainable as a genetic polymorphism; it can only be achieved by a behavioral polymorphism (Maynard Smith 1982). Whereas the mixed ESS predicted for Hawk and Dove games in infinite populations might be realizable as either a genetic or behavioral polymorphism, the outcome in a finite population can be much more variable. Not surprisingly, given our earlier discussion of games in finite populations, a genetic polymorphism, a behavioral polymorphism, or a pure strategy can be the most likely outcome depending upon the population size, levels of selection, and levels of stochasticity (Vickery 1987; Maynard Smith 1988; Vickery 1988; Bergstrom and Godfrey-Smith 1998; Orzack and Hines 2005). The value of f_e in finite populations can vary with the population size (Schaffer 1988), and the most likely mix in a behavioral polymorphism may be skewed away from f_e if the payoff of one pure strategy against itself is much further from the equilibrium payoff than that of the other (Ficici and Pollack 2007).

Useful links

Below, we have listed a few online sites that provide software for simulations, analysis of simple game models, or other tools for studying evolutionary games:

• Dynamo (http://www.ssc.wisc.edu/~whs/dynamo/): A set of evolutionary game theory routines that can be run in the Mathematica environment by W. Sandholm and associates. Visualization tools for simplex analysis are available. Routines are limited to certain combinations of numbers of roles and strategies.
• Gambit (http://www.gambit-project.org/): This is a free program developed by researchers at California Institute of Technology in collaboration with colleagues in the UK and New Zealand. Most versions run on all platforms. The focus is on Nash equilibria and games can be constructed in either normal or extensive form. Strategies must be discrete.
• Social Dynamics Modeling (http://www.socdynamics.org/id4.html): This site provides a long list of social dynamic, including game, models and links to software to study those models. Different software runs in different environments. Popular environments include Matlab, Mathematica, R, Stella, C++, Java, and Excel.
• VirtualLabs Packages (http://www.univie.ac.at/virtuallabs/): This site posted by Christopher Hauert provides no downloadable software, but it does provide a wide variety of simulations and tutorials on different kinds of problems in evolutionary game theory.
• GameTheory Explorer (http://gte.csc.liv.ac.uk/index/): This package provides introductory examples and tools for exploring general game theory questions.
• Bimatrix Game Solutions (http://banach.lse.ac.uk/form.html): This site posted by Rahul Savani provides a GUI in which one can specify the dimensions of a discrete game and enter the relevant payoff matrices. It then finds any equilibria, average payoffs, etc.
• GameBug (http://www.nbb.cornell.edu/Gamebug/): A series of tutorials in evolutionary game theory, covering symmetric and asymmetric games, stable equilibria, probability in payoffs, relatedness, conflict games, and cooperation games, by Robert Wyttenbach, H. Kern Reeve, and Ronald Hoy).

Recommended further reading

Some of the advanced books on evolutionary game theory, following the tradition in economics, intersperse text with exercises whose answers may or may not be provided at the end of the book. Since these exercises are often crucial steps in the general treatment, unanswered exercises can be frustrating to readers who do not have the time or skill to solve them. Also following traditional economics, and perhaps spurred by the initially informal way evolutionary games were defined, many books are organized into series of definitions, theorems, and lemmas, each followed by detailed specification of terms and proofs. Trying to extract the general point given all the proofs can also be challenging, especially when no chapter summaries are provided. Below, we list some of the sources and add our annotations on the “type” of source in order of increasing difficulty:

• Maynard Smith (1982): This is the classic introduction to evolutionary game theory by one of the key players who invented the approach. It is very readable, and although dated in some ways, still constitutes one of the best starting points on the topic. Some algebraic skill is required. See also a follow-up summary by Houston and McNamara (2005).
• Parker (1984): Although cited much less often than Maynard Smith, G. A. Parker played an extremely important role in the early promotion and application of evolutionary game theory approaches. Again, though dated, his chapter cited here is one of the most succinct introductions available. He has also written a fascinating personal account of the history of evolutionary game theory (Parker 2010).
• Nowak (2006): This book is a very accessible introduction to general evolutionary processes in finite populations, the replicator equation, and the application of these ideas to evolutionary games. It also provides an introduction to spatial games. Unlike the prior two sources, this book is the first in this list to focus on evolutionary dynamics instead of end-points. Reading the book first will help readers digest a more recent update on the same topics in Nowak et al. (2010).
• Vincent and Brown (2005): This broad-scope book also invokes some definitions and proofs, but all of the examples are worked out for the reader. The approach focuses on G-functions which define how a player’s fitness depends on its strategy, the current population mix of strategies, the abundances of each strategy, the population size, and amounts of available resources. Derivatives of G-functions are then used like the replicator equation to study evolutionary dynamics and plot trajectories. There is a strong emphasis on interspecific games and community ecology, but also some serious discussion of intraspecific game models. A more recent update on some of the issues raised in the book can be found in McGill and Brown (2007).
• Hofbauer and Sigmund (1998): This book is one of the foundations of evolutionary games based on replicator dynamics. It is heavy on theorems and lemmas, and many critical points are only mentioned as exercises (with no answers provided). Despite this, the book is a much more readable source than their earlier volume on the same topic (Hofbauer and Sigmund 1988).
• Weibull (1998): This was published in the same year as Hofbauer and Sigmund’s second volume. It covers much of the same ground, and is also full of theorems and lemmas, but all the examples are solved. This book is somewhat terse at points, and requires some reasonable algebraic skill, but it can provide a useful counterpoint to or additional perspective on topics in Hofbauer and Sigmund.
• Gintis (2000): In contrast to most of the prior texts, Gintis devotes most of this book to helping the reader learn how to use the tools of game theory, particularly extensive form games. The book is full of exercises, only some of which are solved (in the back). He has designed the text to help readers think like a game theorist and get them involved in the necessary protocols and logic. It helps if readers already know some of the relevant theory. Note that the book is not primarily aimed at evolutionary games although it includes some very useful examples.
• Cressman (2003): This book focuses on the benefits of using extensive form to analyze evolutionary games. A major claim is that the complicated dynamics of higher dimensional games are much more transparent and easily understood using extensive form analysis. The book includes some complicated definitions and theorem proofs, but most examples are solved in the book, not left to the reader. This explicit linking of evolutionary dynamics and extensive form analysis is a very useful perspective.
• Other sources in brief:
  • Thomas (1986): This now-dated book was aimed more at economics than biology but still has some useful insights.
  • Vega-Redondo (1997): This volume is also somewhat dated, but has some very good presentations on basic ESS concepts, replicator dynamics, and stochastic games.
  • Sandholm (2011): This recent text contains a very advanced, and mathematically intense, integration of evolutionary games in large populations with evolutionary dynamics. Convergence stability is of major interest. An innovative classification of different types of population games puts a new spin on the many different game examples that have accumulated over the last decades. Sandholm is also the supervising author for the Dynamo game software mentioned above.
  • Dercole and Rinaldi (2008): These authors review the logic and applications of the canonical equation for adaptive dynamics. Although the focus is more on ecological than behavioral contexts, several sections discuss evolutionary games.
  • Dugatkin and Reeve (1998): This multi-authored book provides an interesting, albeit somewhat dated, set of examples of applications of evolutionary game theory to animal behavior contexts. The chapter by Hammerstein is a succinct summary of the theory at that time, and the chapter by Johnstone provides a useful introduction to evolutionary game models of communication.

Literature cited

Abrams, P.A., H. Matsuda and Y. Harada. 1993. Evolutionarily unstable fitness maxima and stable fitness minima of continuous traits. Evolutionary Ecology 7: 465–487.

Alonzo, S.H. and B. Sinervo. 2001. Mate choice games, context-dependent good genes, and genetic cycles in the side-blotched lizard, Uta stansburiana. Behavioral Ecology and Sociobiology 49: 176–186.

Altrock, P.M. and A. Traulsen. 2009. Deterministic evolutionary game dynamics in finite populations. Physical Review E 80: article 011909.

Apaloo, J., J.S. Brown and T.L. Vincent. 2009. Evolutionary game theory: ESS, convergence stability, and NIS. Evolutionary Ecology Research 11: 489–515.

Apaloo, J. and S. Butler. 2009. Evolutionary stabilities in multidimensional-traits and several-species models. Evolutionary Ecology Research 11: 637–650.

Bach, L.A., T. Helvik and F.B. Christiansen. 2006. The evolution of n-player cooperation ­— threshold games and ESS bifurcations. Journal of Theoretical Biology 238: 426–434.

Bell, G. 1982. The Masterpiece of Nature: the Evolution and Genetics of Asexuality. Berkeley CA: University of California Press.

Bergstrom, C.T. and P. Godfrey-Smith. 1998. On the evolution of behavioral heterogeneity in individuals and populations. Biology and Philosophy 13: 205–231.

Bergstrom, C.T. and M. Lachmann. 1998. Signaling among relatives. III. Talk is cheap. Proceedings of the National Academy of Sciences of the United States of America 95: 5100–5105.

Bergstrom, C.T., S. Szamado and M. Lachmann. 2002. Separating equilibria in continuous signalling games. Philosophical Transactions of the Royal Society of London Series B-Biological Sciences 357: 1595–1606.

Bishop, D.T. and C. Cannings. 1978. Generalized war of attrition. Journal of Theoretical Biology 70: 85–124.

Bishop, D.T., C. Cannings and J.M. Smith. 1978. War of attrition with random rewards. Journal of Theoretical Biology 74: 377–388.

Broom, M., C. Cannings and G.T. Vickers. 1997. Multi-player matrix games. Bulletin of Mathematical Biology 59: 931–952.

Cressman, R. 2009. Continuously stable strategies, neighborhood superiority and two-player games with continuous strategy space. International Journal of Game Theory 38: 221–247.

Cressman, R. and V. Krivan. 2010. The ideal free distribution as an evolutionarily stable state in density-dependent population games. Oikos 119: 1231–1242.

Cresssman, R. 2003. Evolutionary Dynamics and Extensive Form Games. Cambridge, MA: MIT Press.

Dercole, F. and S. Rinaldi. 2008. Analysis of Evolutionary Processes: The Adaptive Dynamics Approach and Its Applications. Princeton NJ: Princeton University Press.

Dieckmann, U. and R. Law. 1996. The dynamical theory of coevolution: A derivation from stochastic ecological processes. Journal of Mathematical Biology 34: 579–612.

Dugatkin, L.A. and H.K. Reeve. 1998. Game Theory and Animal Behavior. New York NY: Oxford University Press.

Eshel, I. 1983. Evolutionary and continuous stability. Journal of Theoretical Biology 103: 99–111.

Eshel, I. and M.W. Feldman. 1984. Initial increase of new mutants and some continuity properties of ESS in two-locus systems. The American Naturalist 124: 631–640.

Eshel, I., U. Motro and E. Sansone. 1997. Continuous stability and evolutionary convergence. Journal of Theoretical Biology 185: 333–343.

Ficici, S.G. and J.B. Pollack. 2007. Evolutionary dynamics of finite populations in games with polymorphic fitness equilibria. Journal of Theoretical Biology 247: 426–441.

Fogel, G.B., P.C. Andrews and D.B. Fogel. 1998. On the instability of evolutionary stable strategies in small populations. Ecological Modelling 109: 283–294.

Frean, M. and E.R. Abraham. 2001. Rock–scissors–paper and the survival of the weakest. Proceedings of the Royal Society of London Series B-Biological Sciences 268: 1323–1327.

Fudenberg, D., M.A. Nowak, C. Taylor and L.A. Imhof. 2006. Evolutionary game dynamics in finite populations with strong selection and weak mutation. Theoretical Population Biology 70: 352–363.

Geritz, S.A.H., E. Kisdi, G. Meszena and J.A.J. Metz. 1998. Evolutionarily singular strategies and the adaptive growth and branching of the evolutionary tree. Evolutionary Ecology 12: 35–57.

Geritz, S.A.H., J.A.J. Metz, E. Kisdi and G. Meszena. 1997. Dynamics of adaptation and evolutionary branching. Physical Review Letters 78: 2024–2027.

Getty, T. 2006. Sexually selected signals are not similar to sports handicaps. Trends Ecol. Evol. 21: 83–88.

Gintis, H. 2000. Game Theory Evolving. Princeton NJ: Princeton University Press.

Grafen, A. 2007. An inclusive fitness analysis of altruism on a cyclical network. Journal of Evolutionary Biology 20: 2278–2283.

Grafen, A. and M. Archetti. 2008. Natural selection of altruism in inelastic viscous homogeneous populations. Journal of Theoretical Biology 252: 694–710.

Hamblin, S. and P.L. Hurd. 2007. Genetic algorithms and non-ESS solutions to game theory models. Animal Behaviour 74: 1005–1018.

Hamilton, W.D. 1971. Geometry for the selfish herd. Journal of Theoretical Biology 31: 295–311.

Hamilton, W.D. 1980. Sex versus non-sex versus parasite. Oikos 35: 282–290.

Hammerstein, P. 1981. The role of asymmetries in animal contests. Animal Behaviour 29: 193–205.

Hammerstein, P. and G.A. Parker. 1982. The asymmetric war of attrition. Journal of Theoretical Biology 96: 647–682.

Hofbauer, J. and K. Sigmund. 1988. The Theory of Evolution and Dynamical Systems. Cambridge UK: Cambridge University Press.

Hofbauer, J. and K. Sigmund. 1998. Evolutionary Games and Population Dynamics. Cambridge UK: Cambridge University Press.

Houston, A.I. and J.M. McNamara. 1999. Models of Adaptive Behaviour: An Approach based on State. Cambridge, UK: Cambridge University Press.

Houston, A.I. and J.M. McNamara. 2005. John Maynard Smith and the importance of consistency in evolutionary game theory. Biology and Philosopnhy 20: 933–950.

Hurd, P.L. 1995. Communication in discrete action-response games. Journal of Theoretical Biology 174: 217–222.

Hurd, P.L. and M. Enquist. 2005. A strategic taxonomy of biological communication. Animal Behaviour 70: 1155–1170.

Krivan, V., R. Cressman and C. Schneider. 2008. The ideal free distribution: A review and synthesis of the game-theoretic perspective. Theoretical Population Biology 73: 403–425.

Lachmann, M. and C.T. Bergstrom. 1998. Signalling among relatives — II. Beyond the tower of Babel. Theoretical Population Biology 54: 146–160.

Lehmann, L., L. Keller and D.J.T. Sumpter. 2007. The evolution of helping and harming on graphs: the return of the inclusive fitness effect. Journal of Evolutionary Biology 20: 2284–2295.

Leimar, O. 2005. The evolution of phenotypic polymorphism: Randomized strategies versus evolutionary branching. The American Naturalist 165: 669–681.

Leimar, O. 2009. Multidimensional convergence stability. Evolutionary Ecology Research 11: 191–208.

Mangel, M. and C.W. Clark. 1988. Dynamic Modeling in Behavioral Ecology. Princeton NJ: Princeton University Press.

Mayard Smith, J. 1974. The theory of games and the evolution of animal conflicts. Journal of Theoretical Biology 47: 209–221.

Maynard Smith, J. 1982. Evolution and the Theory of Games. Cambridge UK: Cambridge University Press.

Maynard Smith, J. 1988. Can a mixed strategy be stable in a finite population. Journal of Theoretical Biology 130: 247–251.

Maynard Smith, J. and G.R. Price. 1973. The logic of animal conflicts. Nature 246: 15–18.

McGill, B.J. and J.S. Brown. 2007. Evolutionary game theory and adaptive dynamics of continuous traits. Annual Review of Ecology Evolution and Systematics 38: 403–435.

Metz, J.A.J., S.A.H. Geritz, G. Meszéna, F.J.A. Jacobs and J.S. van Heerwaarden. 1996. Adaptive dynamics, a geometrical study of the consequences of nearly faithful reproduction. In Stochastic and Spatial Structures of Dynamical Systems (van Strien, S.J. and S.M. Verduyn Lunel, eds). Amsterda, Netherlands: North-Holland Publishing Company. pp. 183–231.

Nash, J.F. 1950. Equilibrium points in n-person games. Proceedings of the National Academy of Sciences of the United States of America 36: 48–49.

Nash, J.F. 1951. Non-cooperative games. Annals of Mathematics 54: 286–295.

Nowak, M.A. 2006. Evolutionary Dynamics: Exploring the Equations of Life. Cambridge, MA: Belknap (Harvard University) Press.

Nowak, M.A. and R.M. May. 1992. Evolutionary games and spatial chaos. Nature 359: 826–829.

Nowak, M.A., A. Sasaki, C. Taylor and D. Fudenberg. 2004. Emergence of cooperation and evolutionary stability in finite populations. Nature 428: 646–650.

Nowak, M.A., C.E. Tarnita and T. Antal. 2010. Evolutionary dynamics in structured populations. Philosophical Transactions of the Royal Society B-Biological Sciences 365: 19–30.

Ohtsuki, H., P. Bordalob and M.A. Nowak. 2007a. The one-third law of evolutionary dynamics. Journal of Theoretical Biology 249: 289–295.

Ohtsuki, H. and M.A. Nowak. 2006. Evolutionary games on cycles. Proceedings of the Royal Society B-Biological Sciences 273: 2249–2256.

Ohtsuki, H. and M.A. Nowak. 2007. Direct reciprocity on graphs. Journal of Theoretical Biology 247: 462–470.

Ohtsuki, H., J.M. Pacheco and M.A. Nowak. 2007b. Evolutionary graph theory: Breaking the symmetry between interaction and replacement. Journal of Theoretical Biology 246: 681–694.

Orzack, S.H. and W.G.S. Hines. 2005. The evolution of strategy variation: Will an ESS evolve? Evolution 59: 1183–1193.

Pacheco, J.M., A. Traulsen and M.A. Nowak. 2006. Active linking in evolutionary games. Journal of Theoretical Biology 243: 437–443.

Parker, G.A. 1974. Assessment strategy and evolution of fighting behavior. Journal of Theoretical Biology 47: 223–243.

Parker, G.A. 1982. Phenotype limited evolutionarily stable strategies. In Current Problems in Sociobiology (Bertram, B.R., T. Clutton-Brock, R.I.M. Dunbar, D.I. Rubenstein and R. Wrangham, eds). Cambridge UK: Cambridge University PRess. pp. 173–201.

Parker, G.A. 1984. Evolutionarily stable strategies. In Behavioural Ecology: An Evolutionary Approach, Second Edition (Krebs, J.R. and N.B. Davies, eds). Oxford UK: Blackwell Scientific Publications. pp. 30–61.

Parker, G.A. 2010. Reflections at dusk. In, Leaders in Animal Behavior: The Second Generation (Drickhamer, L.C. and D.A. Dewsbury, eds). Cambridge UK: Cambridge University Press. pp. 429–464.

Parker, G.A. and D.I. Rubenstein. 1981. Role assessment, reserve strategy, and acquisition of information in asymmetric animal conflicts. Animal Behaviour 29: 221–240.

Rand, D.A., H.B. Wilson and J.M. McGlade. 1994. Dynamics and evolution-evolutionarily stable attractors, invasion exponents, and phenotype dynamics. Philosophical Transactions of the Royal Society of London Series B-Biological Sciences 343: 261–283.

Salathe, M., R.D. Kouyos and S. Bonhoeffer. 2008. The state of affairs in the kingdom of the Red Queen. Trends in Ecology and Evolution 23: 439–445.

Sandholm, W.H. 2011. Population Games and Evolutionary Dynamics. Cambridge, MA: The MIT Press.

Santos, F.C., J.F. Rodrigues and J.M. Pacheco. 2006. Graph topology plays a determinant role in the evolution of cooperation. Proceedings of the Royal Society B-Biological Sciences 273: 51–55.

Schaffer, M.E. 1988. Evolutionarily stable strategies for a finite population and a variable contest size. Journal of Theoretical Biology 132: 469–478.

Seger, J. and H.J. Brockmann. 1987. What is bet-hedging? Oxford Surveys in Evolutionary Biology 4: 182–211.

Sinervo, B., A. Chaine, J. Clobert, R. Calsbeek, L. Hazard, L. Lancaster, A.G. McAdam, S. Alonzo, G. Corrigan and M.E. Hochberg. 2006. Self-recognition, color signals, and cycles of greenbeard mutualism and altruism. Proceedings of the National Academy of Sciences of the United States of America 103: 7372–7377.

Sinervo, B., B. Heulin, Y. Surget-Groba, J. Clobert, D.B. Miles, A. Corl, A. Chaine and A. Davis. 2007. Models of density-dependent genic selection and a new rock–paper–scissors social system. The American Naturalist 170: 663–680.

Strogatz, S.H. 1994. Nonlinear Dynamics and Chaos. Cambridge, MA: Westview Press.

Tarnita, C.E., T. Antal, H. Ohtsuki and M.A. Nowak. 2009a. Evolutionary dynamics in set structured populations. Proceedings of the National Academy of Sciences of the United States of America 106: 8601–8604.

Tarnita, C.E., H. Ohtsuki, T. Antal, F. Fu and M.A. Nowak. 2009b. Strategy selection in structured populations. Journal of Theoretical Biology 259: 570–581.

Taylor, C., D. Fudenberg, A. Sasaki and M.A. Nowak. 2004. Evolutionary game dynamics in finite populations. Bulletin of Mathematical Biology 66: 1621–1644.

Taylor, C. and M.A. Nowak. 2006. Evolutionary game dynamics with non-uniform interaction rates. Theoretical Population Biology 69: 243–252.

Taylor, P.D. and L. Jonker. 1978. Evolutionarily stable strategies and game dynamics. Mathematical Biosciences 40: 145–156.

Thomas, L.C. 1986. Games, Theory, and Applications. New York NY: Halstead Press.

Traulsen, A., J.C. Claussen and C. Hauert. 2005. Coevolutionary dynamics: From finite to infinite populations. Physical Review Letters 95: Article 238701.

Traulsen, A., J.C. Claussen and C. Hauert. 2006a. Coevolutionary dynamics in large, but finite populations. Physical Review E 74: Article 011901.

Traulsen, A., J.M. Pacheco and L.A. Imhof. 2006b. Stochasticity and evolutionary stability. Physical Review E 74: Article 021905.

Traulsen, A., J.M. Pacheco and M.A. Nowak. 2007. Pairwise comparison and selection temperature in evolutionary game dynamics. Journal of Theoretical Biology 246: 522–529.

Vega-Redondo, F. 1997. Evolution, Games, and Economic Behaviour. Oxford UK: Oxford University Press.

Vickery, W.L. 1987. How to cheat against a simple mixed strategy ESS. Journal of Theoretical Biology 127: 133–139.

Vickery, W.L. 1988. Can a mixed strategy be stable in a finite population-reply. Journal of Theoretical Biology 132: 375–378.

Vincent, T.L. and J.S. Brown. 1988. The evolution of ESS theory. Annual Review of Ecology Evolution and Systematics 19: 423–443.

Vincent, T.L. and J.S. Brown. 2005. Evolutionary Game Theory, Natural Selection, and Darwinian Dynamics. Cambridge UK: Cambridge University Press.

Weibull, J.W. 1998. Evolutionary Game Theory. Cambridge, MA: The MIT Press.

Weissing, F.J. 1991. Evolutionary stability and dynamic stability in a class of evolutionary normal form games. In Game Equilibrium Models. I. Evolution and Game Dynamics (Selten, R., eds). Berlin Germany: Springer-Verlag. pp. 29–97.

Zhou, D., B. Wu and H. Ge. 2010. Evolutionary stability and quasi-stationary strategy in stochastic evolutionary game dynamics. Journal of Theoretical Biology 264: 874–881.

10.6 Handicap Controversies

The Handicap Principle

Amotz Zahavi first developed the handicap model of signal evolution in 1975 as an alternative to R. A. Fisher’s (1958) runaway model of mate choice (Zahavi 1975, 2003). The question was, how can extremely extravagant sexually dimorphic traits sometimes evolve in males, when those traits clearly impose a significant cost to the bearer (i.e., the long tail of the male peacock)? Fisher’s argument was that initially, females would evolve a preference for a male trait that was associated with male fitness or quality. For example, larger males need longer tails for flying and steering, so a female preference for longer tails would result in them obtaining larger, better quality mates. Once many females acquired this preference, males could benefit from increasing their tail length beyond what was required for flying. The inefficiency cost of bearing a longer tail was compensated by the attraction of more mates. The more females preferred males with longer tails, the more males benefited by growing even longer tails, initiating a runaway process. Long tails in males and female preference for long tails would become genetically linked. The runaway process would come to a halt when the cost of further tail elongation became too great for males to bear. Fisher assumed that once all males invested in growing their tails, tail length was no longer correlated to their quality. Females maintained their preference for the tails in order to produce sons that would be successful and preferred by the next generation of females.

Zahavi countered that sexual selection via female choice should be viewed as a sender–receiver signaling interaction, where males strive to advertise their quality and females are selected to pay attention only to those signal traits that honestly reveal male quality. Rather than a runaway process that ends with no correlation between tail length and male quality, he argued that costly tails could help to reveal differences in male quality because better quality males can carry a heavier burden than lower quality males. Under sexual selection, males would thus evolve costly signaling traits as a test of their quality to discriminating females. Zahavi called this idea the handicap principle, deriving the term from the analogy of handicapping faster racehorses with heavier weights. Female receivers benefit by selecting males with handicaps because they can be assured of the quality of their mates, while male senders benefit because they can better advertise their quality and thus acquire more or better female mates. But both sexes also lose: males lose by investing time and energy in advertising, and females may receive less help from their mates and bear sons which are less fit to withstand the pressures of natural selection. But an individual without the handicapping marker ­does not advertise its quality, so a potential mate cannot detect it. Moreover, the particular marker must handicap the specific aspect of quality that is important to the receiver. Thus selection for the handicap creates a biological link between the information encoded in the signal and the form of the signal trait. In Zahavi’s words, “a rich person can signal the degree of his wealth by wasting money. His signal is reliable since a poorer man cannot waste as much money. A courageous man can display the degree of his courage by taking a risk, which a less courageous individual would not dare to take. However, taking a risk of bodily harm does not display wealth, and spending money does not display how brave a person is.” Figures 1 and 2 show two animal examples of possible handicap signals.

Figure 1: A foraging handicap? Both the male and female American white pelican (Pelecanus erythrorhynchos) develop a fibrous, dorso-laterally flattened structure on the bill (called the “maxillary horn”) during the annual courtship period. Zahavi speculates that the horn interferes with the ability of the bird to see around the tip of the bill, requiring a forager to remember where it last saw a prey fish and project its likely movements. An inexperienced bird would be less able to do this, so horn size reflects the foraging ability of a bird (Zahavi & Zahavi 1997). We don’t know whether the horn impairs vision and foraging. Like many other pelican species, the white pelican forages by swimming on the surface and jabbing its head under the water to scoop up small fish in the flexible pouch on the underside of the bill. The horn is absent in young birds and larger in paired breeding birds, presumably because they are older, so its size is correlated with age. The horn must impose some kind of cost because it is shed as soon as mating is finished. Aggressive bill jabbing and holding is often directed at this part of the opponent’s bill, so another possibility is that the condition of the horn reflects fighting ability (Knopf & Evans 2004). Although it looks like a handicap, there is no proof whatsoever that it really is a handicap signal. (Photo courtesy of Tripp Davenport.)

Figure 2: Lateral compression threat displays in lizards. As discussed in the main text, while performing these broadside lateral displays, the male compresses his ribs, which impairs respiration. The male is literally holding his breath! The reduction in aerobic metabolism causes a compensatory increase in anaerobic metabolism, which generates lactate and diminishes locomotory capacity. Only males in good condition can afford this temporary reduction in aerobic capacity and still mobilize the energy to fight. The natural duration of time the posture is held is correlated with the male’s intrinsic endurance on a treadmill, and forced exercise reduces subsequent display duration. The display thus expends the quality attribute being advertised (Brandt 2003). This is a much more convincing example of a handicap than the pelican in Figure 1. (Photo of Sceloporus occidentalis courtesy of Yoni Brandt.)

Controversy over the handicap term

The handicap term itself is deemed to be unfortunate, even by Zahavi himself: “…the word ‘handicap’ is misleading because it has the connotation of a loss. Senders are not losing – they invest in order to gain: an individual that takes on a reasonable handicap in order to signal is like a businessman investing in advertisement” (Zahavi 2007b). Getty is more adamant, pointing out that in the sports analogy, the function of weight handicaps is to even out the performance of different quality competitors (Getty 2006). The handicap principle, on the other hand, argues that the selective pressure exerted by receivers for increasingly extravagant display traits enhances the variation in signal trait expression among senders and makes it easier for receivers to discern true differences in sender quality.

Another controversial expression used by Zahavi to describe signal evolution is selection for “waste” and “inefficiency” (Zahavi 1991). According to Zahavi, non-signal characters are selected for efficiency, and the smaller the investment required to achieve a particular result the better. By contrast, signals must have a cost to be reliable indicators of quality, and the greater the investment, the greater the reliability. Strong selection by receivers for increasingly costly and reliable signals can lead to extravagant signal traits that seem wasteful and inefficient. Furthermore, the nature of the “wasted” commodity is a key to the information encoded in the signal. While the need for costliness may be unique to signal traits, all traits can be expected to have some cost and to have tradeoffs with other survival and reproductive fitness components. John (1997) further points out that “costliness is not synonymous with inefficiency (which alludes to net costliness instead). From the gene’s point of view, costly signals are neither wasteful nor inefficient if they maximize fitness.” Thus terms like handicap and inefficiency, while colorful descriptors of some aspects of signal evolution, have tended to exaggerate and confuse the underlying principle of the need for some form of cost trade-off that makes cheating a suboptimal strategy compared to honesty.

Controversy over verbal versus math models

Zahavi’s verbal handicap model, along with the controversial terms sometimes used to describe it, immediately stimulated several evolutionary theoreticians to attempt to model it quantitatively. They believed that mathematical models made the assumptions of a model clearer than verbal arguments and could show more rigorously whether or not the suggested conclusions were correct. These early models concluded that handicap signals could not evolve as proposed (Davis & O’Donald 1976; Maynard Smith 1976; Bell 1978; Kirkpatrick 1986). But the theoreticians misunderstood the gist of the argument, and Zahavi’s admittedly confusing graphical representation of it. These early models incorrectly assumed the costs of the signal lowered the fitness of all males in the same proportion. Parker (1979) subsequently showed that either the costs of signaling, or the potential benefits obtained when receivers respond favorably, had to be differentially scaled for different senders. These alternatives became known as the “differential costs,” or quality handicap model, and the “differential benefits,” or signal of need versions of the handicap model. When signaling costs differentially penalize either lower quality or less needy senders, reliable signals can be maintained (Pomiankowski 1987; Grafen 1990a, 1990b; Iwasa & Pomiankowski 1991; Johnstone 1995, 1997). Maynard Smith quickly accepted the basic tenet of the handicap principle (1991) and publicly apologized to Zahavi at a meeting of the Royal Society in 1992, admitting that he had been wrong.

Additive versus multiplicative models

Grafen’s differential cost model requires that higher-quality senders pay lower marginal costs for advertising in order to maintain stable honest signaling. But this requirement is correct only if the fitness benefits and costs are strictly additive, i.e., measured in the same currency (Figure 3).

Figure 3: Additive model. The green line indicates benefit to the sender for expressing a signal size of a, and the three blue lines show the cost of the signal for low, medium, and high quality senders. The marginal cost of a signal at any given signal size is the slope of the cost line. The optimal signal size for each quality class occurs where the marginal cost equals the marginal benefit (slopes are equal) and the net benefit (B–C) is maximal, as shown by the orange vertical lines.

However, additivity is not expected from most life history models, where benefit is measured in terms of mating success and cost is measured in terms of viability (or mortality rate). These components must be multiplied to obtain net fitness. A more general criterion for honest handicap signaling is that higher-quality senders must have higher marginal fitness effects of advertising. This effect could arise from higher fitness benefits as well as lower costs. The general criterion implies the existence of a ridge on the fitness surface for two correlated characters, quality (or viability) and signal intensity (Figure 4). In a multiplicative model, higher quality senders can have higher marginal viability and fitness costs and still be more efficient because of their higher marginal fitness benefits (Getty 1998, 2006). Thus Getty further argued that the additive concept of the handicap term was technically incorrect, metaphorically misleading and a poor guide for empirical research on signal evolution. He also argued that signal evolution should not be viewed as a “missing piece of Darwin’s puzzle”, but rather as an integral piece of the process of evolution by natural selection.

Figure 4: Multiplicative model. The green line is fecundity or mating advantage F for a signal size of a. The three blue lines show the residual viability V as a function of a for low, medium, and high quality senders. The cost of the signal is the drop in residual viability, and the marginal viability cost is the steepness of the negative slope. Fitness is the product of F and V represented by the orange lines, with the optimal value indicated by orange circles. In (A), the marginal viability cost is higher for higher quality senders, which is inconsistent with the Grafen model, but could happen if there were a power-efficiency (i.e., intercept-slope) tradeoff as shown in the graph. The optimal signal size increases with sender quality despite the violation of the Grafen requirement. In (B), the marginal viability costs are lower for higher quality senders, consistent with the Grafen requirement. The optimal signal size increases with sender quality. But at equilibrium, the high-quality senders are not “wasting” more viability, they are actually more efficient than low-quality senders. (After Nur & Hasson 1984; Getty 2006.)

Are all signals handicaps?

Zahavi views all signals, and some characters and behaviors that we would at first not even think of as signals, through the lens of the handicap principle (Zahavi & Zahavi 1997; Zahavi 2007a, b, c). If the principle is defined extremely broadly, such that any type of cost is a “handicap,” then he would often be right. But a very broad view that places an umbrella over everything prevents us from exploring some interesting variation in the process of evolving signals that encode different kinds of information in different ecological and social contexts. For example, Zahavi removes the physiological constraint distinctiveness of an unfakable index signal, compared to the cost–benefit tradeoff a quality handicap signal, by stating that all handicaps are indices with an analog relationship between trait value and sender quality (Zahavi 2007; unpublished manuscript). Zahavi dismisses the idea of arbitrary conventional signals in conflict of interest contexts that are honest as a result of a receiver retaliation rule. Status badges, for example, are designed to give the illusion of excellence. Thus the vertical breast strip in great tit makes the breast look narrower than it is, and only a high-quality individual can afford the narrowing effect of a wider band. Zahavi proposes that a reliable threat must leave the sender open to attack. Displays that transmit information about the sender’s confidence, motivation, or willingness to take risks should thus expose vulnerable body parts or assume postures that make it difficult to launch an attack. He suggests that the broadside threat display common in fish, antelope, and lizards exposes the flank of the sender and leaves any weapons on the front of the animal in an inconvenient position to attack. Threats that involve stretching some part of the body, with back arching, fins, or fur, are not likely to deceive the opponent about the sender’s body size, but place the sender in postures from which flight or attack are difficult, and provide a structure that the opponent can grab (Zahavi 1981; Zahavi & Zahavi 1997). Zahavi further asserts that amplifiers, such as lines on the body that make assessment of body size easier, cannot be separated from body size and are handicaps that selectively penalize smaller individuals. Finally, Zahavi claims that there is essentially always a conflict of interest between any two individuals, even closely related ones, and to be on the safe side, signals should always have an honesty-guaranteeing cost. Low-cost or cost-free signals should rarely or never occur (Zahavi 1993, 2003).

Számadó takes this umbrella approach to task in his review of handicap signals entitled “The cost of honesty and the fallacy of the handicap principle” (2011). In agreement with our approach in the main text, handicap signals are restricted to signals for which a realized strategic cost can be demonstrated, and where there is a biological constraint that links the size, quality, or intensity of the display to the attribute of interest to the receiver. Proulx (2001) explicitly modeled the relationship between sender’s display costs and utility (fitness advantages) of the advertised quality to the receiver; when utility and quality are decoupled, no signaling equilibrium can be attained. Other types of signals that do not entail this realized cost should not be called handicaps. For instance, signals for which the honest sender does not pay the extra strategic cost at equilibrium, only paying if it cheats, involves a potential cost. Conventional threat signals with a receiver retaliation rule, and proximity threats involving close approach to the rival, are examples of potential cost that are only paid if a weak sender sends a stronger signal than it can support. Similarly, index signals may entail an efficacy cost, but no added strategic cost that guarantees honesty. Signals with pooling equilibria, where senders within broad ranges of qualities give signals of the same intensity, can in theory lead to partially informative low-cost signals (Bergstrom & Lachmann 1998). Finally, signals for which sender and receiver have shared or overlapping interests, such as alarm calls and identity signals, are often cost-free.

Altruism as a handicap signal

Acts such as feeding offspring that are not one’s own, perching as a sentinel while others forage, or mobbing predators, have been traditionally viewed as examples of altruism. By definition, an altruistic act benefits other individuals at some cost to the donor. Traditional explanations for these behaviors range from kin selection to various forms of reciprocity (see textbook Figure 13.1 and Web Topic 13.1). Zahavi has proposed that such behaviors are instead costly handicap signals indicating quality and propensity to cooperate (Zahavi 1977, 1990, 1995). Highly altruistic individuals gain social prestige in the opinion of others, which Zahavi argues benefits the altruist by increasing its attractiveness as a mate or collaborator. Roberts (1998) expanded on this idea that altruism could serve as a competitive display of quality. Lotem et al. (1999) and Putland (2001) emphasized the role of intersexual selection in biparental species with altruistic helping behavior, whereby females prefer as mates former helper males that demonstrate or “show off” their parental ability. These various versions of the social prestige hypothesis fall into the class of cooperative mechanisms called “indirect positive pseudo-reciprocity” (see Bergmüller et al. 2007; Connor 2007; Bshary & Bergmüller 2007; and Web Topic 13.1) because the altruist receives an eventual benefit via its effect on third party receivers . Note that social prestige is not the same thing as dominance status. Dominance involves obtaining benefits via aggression, e.g., gaining priority of access to food and mates by intimidating rivals. Prestige and dominance could generate different hierarchies within groups, but it is likely that the two hierarchies are somewhat correlated since dominants are often in better condition and better able to afford the cost of bestowing altruism (Wright 1999, 2007; Barclay & Reeve 2012).

As with other explanations for cooperation, the evolutionary stability of social prestige mechanisms has been examined theoretically. Quantitative models demonstrate that, in principle, costly altruistic acts can become stable signals of quality. Earlier models of indirect reciprocity had demonstrated that stable cooperation could evolve in social groups if most individuals are known to each other. This condition is also necessary for the building of a reputation as a cooperator. Cooperators might then benefit over the long term because others are more likely to trust and collaborate with them (Boyd & Richerson 1989; Milinski et al. 2002; Nowak & Sigmund 2005). Such models differ from direct reciprocity and conditional altruism (tit-for-tat) models, which require the initial altruist to be repaid later by the same individual it had aided. Indirect pseudo-reciprocity allows reputation (also called “image score”) to be built by “word of mouth” or eavesdropping, and the resulting altruism is unconditional (not dependent on what the beneficiary did at a prior meeting). Lotem et al. (2003) expanded the existing indirect reciprocity model into a signaling model by first introducing individual variation in quality. When the probability of repeated requests for aid is sufficiently common, and high-quality individuals can bear the cost of an altruistic act better than low-quality individuals, then an ESS can evolve in which only high-quality individuals engage in tit-for-tat reciprocal exchanges and low-quality individuals defect. Altruistic behavior thus signals high quality. Adding a signaling benefit component to this game means that altruists gain benefits in contexts other than those in which they performed an altruistic act. When this is true, reputation or prestige causes unconditional altruism to be stable at least over a given set of parameter values. Specifically, when the cost of altruistic acts is sufficiently high, so that low quality individuals cannot obtain a net benefit from performing them, then altruism can serve as a stable handicap signal of quality. Similar quantitative models lend further support to these conclusions (Gintis et al. 2001; Pilot 2005). Thus costly signaling of quality via altruistic acts and the build-up of prestige within groups could be one of several valid explanations for the evolution of altruism within cooperating groups. This explanation avoids the restrictive requirements of kin selection (relatedness), group selection (highly structured populations and low dispersal rates), and direct reciprocity (delayed return of a similar altruistic act by the recipient), but does require individual recognition (or memory of rendezvous sites), a social network, and memory of past acts by others (Nowak 2006).

What types of evidence would be needed to demonstrate that altruism serves as a signal of social prestige, and to distinguish this social prestige mechanism for the evolution of helping behavior from alternative mechanisms? First, one needs to provide evidence of signaling to third party receivers. For instance, the helping behavior could be accompanied by conspicuous signals, a helper’s behavior could differ depending on the presence or absence of an audience, or competitive helpers could be shown to vie for the attention of bystanders. If these effects cannot be shown, and instead, helper behavior functions only to promote the condition or survival of the recipient offspring, then alternative hypotheses for the helping behavior are more likely (Wright 1999, 2007). For example, if improved survival of the offspring primarily increases only the helper altruist’s inclusive fitness component, then the kin selection hypothesis (Hamilton 1964) would be the most likely explanation; if improved offspring survival leads to an increase in group size and enhanced ability to defend the territory, gain more group resources, or disperse more successfully, then the group augmentation hypothesis (Kokko et al. 2001) might be a sufficient explanation for the behavior. Second, given that some evidence of signaling has been found, additional evidence concerning the type of benefit obtained by the altruist is required, because there are two alternative signaling hypotheses—the social prestige hypothesis and the pay-to-stay hypothesis. The pay-to-stay hypothesis proposes that subordinate group members “pay rent” in order to be allowed to remain in the group and potentially gain breeding positions later in life (Gaston 1978; Kokko et al. 2002). Evidence that dominant territory owners punish or evict helpers that slack off in their helping effort, and that harder-working helpers obtain survival and/or breeding status benefits, would support this alternative pay-to-stay hypothesis. Suppose that a careful study does not support a pay-to-stay, enhanced group size, or kin assistance explanation for altruistic acts and that conspicuous signaling when audiences are present is noted. To demonstrate support for the social prestige hypothesis, one would also need to show that: (1) altruists eventually obtain a benefit by being selected by a mate or collaborator; (2) third party receivers do evaluate potential mates/collaborators on the basis of their helping effort; and (3) these third party receivers prefer the better-performing altruists. Real systems are complicated by the fact that the hypotheses for helping listed above (kin selection, group augmentation, pay-to-stay, and social prestige) are not mutually exclusive, so analyses might require a quantitative ranking of their relative contributions to the evolution of the helping behaviors.

What evidence has been obtained for the social prestige hypothesis? The question has been primarily investigated in cooperatively breeding species, but many of these systems involve closely related individuals where the role of kin selection cannot be excluded. Luckily, a few studies have been able to focus on cases of unrelated reproductive helpers and others on the interspecific mutualistic interactions of cleaner fish and their clients. We describe some of these studies below.

Evidence of a signaling role for helping behavior has been described in the colonial and cooperatively breeding sociable weaver (Philetarius socius) (Doutrelant & Covas 2007). Here, the levels of helping behavior are enhanced by the presence of an audience, and this increases the helper’s likelihood of being observed feeding offspring. Specifically, helpers spent a longer time than parents holding the prey at the edge of the colony before feeding, and they waited for the number of third party observers to increase before feeding. However, a benefit for these behaviors has not yet been identified in this species, so social prestige and pay-to-stay hypotheses cannot be distinguished. In another colonial cooperative breeder, the pied kingfisher (Ceryle rudis), good evidence of the pay-to-stay mechanism has been found in the form of punishment to slackers and a subsequent mating advantage to more altruistic male helpers (Reyer 1984, 1990). Similar evidence also supports the pay-to-stay hypothesis in superb fairy-wrens (Malurus cyaneus) (Mulder & Langmore 1993), and the cichlid fish Neolamprologus pulcher (Balshine-Earn et al. 1998, Bergmüller & Taborsky 2005; Bergmüller et al. 2005). However, whether an additional social prestige effect is present has not been excluded.

Zahavi and co-workers (Carlisle & Zahavi 1986; Zahavi & Zahavi 1997; Kalishov et al. 2005) describe how group-living Arabian babblers (Turdoides squamiceps) compete with one another to perform feeding and sentinel acts and sometimes interfere with the altruistic endeavors of others (although these observations were mainly seen during a year with an excessive number of one-year-old helpers). Dominant individuals generally perform more provisioning, guarding, and mobbing activities than subordinate individuals. Dominants usually refuse to be fed by subordinates by walking away from them, or interfere with aid-giving gestures directed toward them by subordinates. Willing recipients of aid, on the other hand, adopt a specific crouched, begging-like posture. These observations are suggestive of a social prestige signaling component for provisioning. Wright and colleagues attempted to test the hypothesis more rigorously by looking for evidence of audience effects, as well as evidence that helpers used different rules for feeding nestlings and serving as sentinels than breeders. There was no evidence of differential competitive or conspicuousness-enhancing behaviors by helpers of different ages, sexes, and statuses, and they also used the same provisioning rules as breeders, which depended on age of nestlings and size of the group (Wright 1997, 1998a, b). Sentinel duty was most often performed by the breeding male and conformed to state-dependent models of individually selfish anti-predator strategies (Wright et al. 2001a, b, c). Thus evidence for prestige signaling in this species remains circumstantial. Dominant breeders who benefit the most from brood survival expend the most aid-giving effort (i.e., straightforward parental investment). The more likely hypotheses for helping in this species are kin selection and perhaps group augmentation. If there is some role for social prestige signaling, then the prestige and dominance hierarchies are concordant and their relative roles difficult to untangle (Wright 1999, 2007).

Another well-studied avian helper-at-the-nest species is the Australian bell miner (Manorina melanophrys), which breeds in dense neighborhoods where non-reproductive helpers (primarily males) and male breeders regularly feed related and unrelated nestlings in a number of nests. Helpers and males also frequently join together to mob predators in the area. If a female loses her mate, she selects the male helper that performed the most provisioning at her recent nests as her new mate, strong evidence of a benefit to vigorously feeding helpers (Clarke 1989). Helpers also give individually distinctive calls when visiting nests, thereby advertising their aid-giving acts. However, careful observational and experimental studies to look for an audience effect on helper behavior found no evidence for this important prediction of the social prestige signaling hypothesis (McDonald et al. 2001a, b). This lack of evidence does not necessarily exclude the operation of a prestige-signaling system in this species, since the other key pre-requisites are present; helpers advertise vocally and so may not require that a female breeder audience is in view. Cooperation in this species may thus be driven by a combination of kin selection, group augmentation, and indirect pseudo-reciprocity.

The cleaner wrasse–client fish mutualistic interaction provides the strongest evidence for an altruist signaling system (Bshary & Schaffer 2002; Grutter & Bshary 2003; Bshary & Grutter 2006; Pinto et al. 2011). The cleaner fish (Labroides dimidiatus) may cooperate and remove ectoparasites from client fish or they may cheat by feeding on client mucus. The cleaner fishes prefer mucus, so refraining from this behavior constitutes an altruistic act. Eavesdropping clients spend more time next to known cooperative cleaners than unknown cleaners, and they respond to any occurrence of cleaner cheating by avoiding them. These observations demonstrate that client fish score and choose individual cleaners on the basis of the quality of their service. Trained cleaners also learn to feed more cooperatively (contrary to their preference) when in a “client-feedback” than in a “non-client-feedback” situation. Finally, cleaners immediately increase their level of cooperative behavior in the presence of a bystander client fish. Cleaners thus benefit in terms of many future clients if they display altruistic behavior in the presence of a bystander client.

Chemical signals as handicaps

Zahavi points out that chemical signals between individuals also require investment in reliability, just like other signal modalities. The cost or handicap may be the ability of the sender to bear the damage caused by the signaling chemical, or it may be the difficulty of producing a particular chemical. Carotenoids used in the production of color signals are proposed as an example of ability to bear the toxic cost of a chemical signal. Circulating carotenoids are beneficial in small amounts because they deactivate harmful oxygenated radicals (anti-oxidant). However, in larger amounts carotenoids may increase the lifetime of radicals (pro-oxidant), such that only high quality individuals may be able to bear this cost (Haila et al. 1996; McGraw et al. 2005). An example of signals that are hard to produce may be the mating pheromones of yeast cells, complex molecules such as glycoproteins that require special investment for their synthesis (Nahon et al. 1995). The alpha mating peptide of yeasts is produced from a complex glycoprotein pro-peptide. Short peptides may not display the phenotypic quality of the secreting sender because they are easy to produce, whereas large proteins with post-translational modifications may vary in relation to the phenotypic quality of the sender. Several other examples of chemical blends and pheromone concentrations that are correlated with mating success and sender status have been described (Iyengar & Eisner 1999; Endler et al. 2006), but more work needs to be done to determine the mechanism by which chemical signals provide honest information. Whether these kinds of chemical signals involve a costly handicap is not at all clear.

Zahavi goes even further to suggest that chemical signaling within multicellular bodies also may require investment in reliability, even though all cells within an organism share the same genome and have no conflict of interest (Zahavi 2008; Zahavi & Perel 2011). The reason why reliability may be needed is to avoid signaling by cell phenotypes that should not signal, or to inhibit the signaling cells from producing too much of the signal. Costly chemical handicaps could guarantee these requirements. Zahavi suggests that within-organism chemical signaling does not function to instruct the receiving cell to take certain actions, but to provide information about the quality or state of the signaling cell. The response of the receiver cell may depend on the phenotypic quality of the sender cell, and receiver cells of different qualities may respond differentially. Examples of costly or toxic chemicals used in between-cell signaling are NO, CO, and steroids. Zahavi claims that cell biologists and endocrinologists are not asking the right questions about cell–cell interactions, and offers this creative alternative view from a between-animal communication perspective.

Literature cited

Balshine-Earn, S., F.C. Neat, H. Reid & M. Taborsky. 1998. Paying to stay or paying to breed? Field evidence for direct benefits of helping behavior in a cooperatively breeding fish. Behavioral Ecology 9: 432-438.

Barclay, P. & H.K. Reeve. 2012. The varying relationship between helping and individual quality. Behavioral Ecology 23: 693-698.

Bell, G. 1978. Handicap principle in sexual selection. Evolution 32: 872–885.

Bergmüller, R. & M. Taborsky. 2005. Experimental manipulation of helping in a cooperative breeder: helpers ‘pay to stay’ by pre-emptive appeasement. Animal Behaviour 69: 19-28.

Bergmüller, R., D. Heg & M. Taborsky. 2005. Helpers in a cooperatively breeding cichlid stay and pay or disperse and breed, depending on ecological constraints. Proceedings of the Royal Society B-Biological Sciences 272: 325-331.

Bergmüller, R., R.A. Johnstone, A.F. Russell & R. Bshary. 2007. Integrating cooperative breeding into theoretical concepts of cooperation. Behavioural Processes 76: 61-72.

Bergstrom, C.T. & M. Lachmann. 1998. Signaling among relatives. III. Talk is cheap. Proceedings of the National Academy of Sciences of the United States of America 95: 5100–5105.

Boyd, R. & P.J. Richerson. 1989. The Evolution of indirect reciprocity. Social Networks 11: 213–236.

Brandt, Y. 2003. Lizard threat display handicaps endurance. Proceedings of the Royal Society of London Series B-Biological Sciences 270: 1061–1068.

Bshary, R. & D. Schaffer. 2002. Choosy reef fish select cleaner fish that provide high-quality service. Animal Behaviour 63: 557-564.

Bshary, R. & A.S. Grutter. 2006. Image scoring and cooperation in a cleaner fish mutualism. Nature 441: 975–978.

Carlisle, T.R. & A. Zahavi. 1986. Helping at the nest, allofeeding and social status in immature Arabian babblers. Behavioral Ecology and Sociobiology 18: 339–351.

Clarke, M.F. 1989. The pattern of helping in the bell miner (Manorina melanophrys). Ethology 80: 292-306.

Connor, R. 2007. Invested, extracted and byproduct benefits: A modified scheme for the evolution of cooperation. Behavioural Processes 76: 109-113.

Davis, J.W.F. & P. O’Donald. 1976. Sexual selection for a handicap—critical analysis of Zahavi’s model. Journal of Theoretical Biology 57: 345–354.

Doutrelant, C. & R. Covas. 2007. Helping has signalling characteristics in a cooperatively breeding bird. Animal Behaviour 74: 739-747.

Endler, A., J. Liebig & B. Hölldobler. 2006. Queen fertility, egg marking and colony size in the ant Camponotus floridanus. Behavioral Ecology and Sociobiology 59: 490–499.

Fisher, R.A. 1958. The Genetical Theory of Natural Selection. London: Clarendon Press.

Gaston, A.J. 1978. Ecology of common babbler Turdoides caudatus. Ibis 120: 415-432.

Getty, T. 1998. Handicap signalling: when fecundity and viability do not add up. Animal Behaviour 56: 127–130.

Getty, T. 2006. Sexually selected signals are not similar to sports handicaps. Trends in Ecology & Evolution 21: 83–88.

Gintis, H., E.A. Smith & S. Bowles. 2001. Costly signaling and cooperation. Journal of Theoretical Biology 213: 103–119.

Grafen, A. 1990a. Sexual selection unhandicapped by the Fisher process. Journal of Theoretical Biology 144: 473–516.

Grafen, A. 1990b. Biological signals as handicaps. Journal of Theoretical Biology 144: 517–546.

Grutter, A.S. & R. Bshary. 2003. Cleaner wrasse prefer client mucus: support for partner control mechanisms in cleaning interactions. Proceedings of the Royal Society B-Biological Sciences 270: S242-S244.

Haila, K.M., S.M. Lievonen & M.I. Heinonen. 1996. Effects of lutein, lycopene, annatto, and gamma-tocopherol on autoxidation of triglycerides. Journal of Agricultural and Food Chemistry 44: 2096–2100.

Hamilton, W.D. 1964. The genetical evolution of social behaviour. Journal of Theoretical Biology 7: 1-52.

Iwasa, Y. & A. Pomiankowski. 1991. The evolution of costly mate preferences. 2. The handicap principle. Evolution 45: 1431–1442.

Iyengar, V.K. & T. Eisner. 1999. Female choice increases offspring fitness in an arctiid moth (Utetheisa ornatrix). Proceedings of the National Academy of Sciences of the United States of America 96: 15013–15016.

John, J.L. 1997. Handicap signal selection is not selection for inefficiency. Animal Behaviour 54: 225–227.

Johnstone, R.A. 1995. Sexual selection, honest advertisement and the handicap principle—reviewing the evidence. Biological Reviews of the Cambridge Philosophical Society 70: 1–65.

Johnstone, R.A. 1997. The evolution of animal signals. In Behavioural Ecology: an Evolutionary Approach (Krebs, J.R. & N.B. Davies, eds.). Oxford: Blackwell Scientific Publications. pp. 155–178.

Kalishov, A., A. Zahavi & A. Zahavi. 2005. Allofeeding in Arabian babblers (Turdoides squamiceps). Journal of Ornithology 146: 141–150.

Kirkpatrick, M. 1986. The handicap mechanism of sexual selection does not work. American Naturalist 127: 222–240.

Knopf, F.L. & R.M. Evans. 2004. American White Pelican (Pelecanus erythrorhynchos). In The Birds of North America Online (Poole, A., eds.). Ithaca: Cornell Laboratory of Ornithology, No. 057.

Kokko, H., R.A. Johnstone & T.H. Clutton-Brock. 2001. The evolution of cooperative breeding through group augmentation. Proceedings of the Royal Society B-Biological Sciences 268: 187-196.

Kokko, H., R.A. Johnstone & J. Wright. 2002. The evolution of parental and alloparental effort in cooperatively breeding groups: when should helpers pay to stay? Behavioral Ecology 13: 291-300.

Lotem, A., R.H. Wagner & S. Balshine-Earn. 1999. The overlooked signaling component of nonsignaling behavior. Behavioral Ecology 10: 209-212.

Lotem, A., M.A. Fishman & L. Stone. 2003. From reciprocity to unconditional altruism through signalling benefits. Proceedings of the Royal Society of London Series B-Biological Sciences 270: 199–205.

Maynard Smith, J. 1976. Sexual selection and handicap principle. Journal of Theoretical Biology 57: 239–242.

Maynard Smith, J. 1991. Theories of sexual selection. Trends in Ecology & Evolution 6: 146–151.

McDonald, P.G., A.J.N. Kazem, M.F. Clarke & J. Wright. 2008a. Helping as a signal: does removal of potential audiences alter helper behavior in the bell miner? Behavioral Ecology 19: 1047-1055.

McDonald, P.G., L. Te Marvelde, A.J.N. Kazem & J. Wright. 2008b. Helping as a signal and the effect of a potential audience during provisioning visits in a cooperative bird. Animal Behaviour 75: 1319-1330.

McGraw, K.J., G.E. Hill & R.S. Parker. 2005. The physiological costs of being colourful: nutritional control of carotenoid utilization in the American goldfinch, Carduelis tristis. Animal Behaviour 69: 653–660.

Milinski, M., D. Semmann & H.J. Krambeck. 2002. Donors to charity gain in both indirect reciprocity and political reputation. Proceedings of the Royal Society of London Series B-Biological Sciences 269: 881–883.

Mulder, R.A. & N.E. Langmore. 1993. Dominant males punish helpers for temporary defection in superb fairy-wrens. Animal Behaviour 45: 830-833.

Nahon, E., D. Atzmony, A. Zahavi & D. Granot. 1995. Mate selection in yeast—a reconsideration of the signals and the message encoded by them. Journal of Theoretical Biology 172: 315–322.

Nowak, M.A. 2006. Five rules for the evolution of cooperation. Science 314: 1560–1563.

Nowak, M.A. & K. Sigmund. 2005. Evolution of indirect reciprocity. Nature 437: 1291–1298.

Nur, N. & O. Hasson. 1984. Phenotypic plasticity and the handicap principle. Journal of Theoretical Biology 110: 275–297.

Parker, G.A. 1979. Sexual selection and sexual conflict. In Sexual Selection and Reproductive Competition in Insects (Blum, M.S. & N.A. Blum, eds.). New York: Academic Press. pp. 123–166.

Pilot, M. 2005. Altruism as an advertisement—a model of the evolution of cooperation based on Zahavi’s handicap principle. Ethology Ecology & Evolution 17: 217–231.

Pinto, A., J. Oates, A. Grutter & R. Bshary. 2011. Cleaner wrasses Labroides dimidiatus are more cooperative in the presence of an audience. Current Biology 21: 1140-1144.

Pomiankowski, A. 1987. Sexual selection—the handicap principle does work sometimes. Proceedings of the Royal Society of London Series B-Biological Sciences 231: 123–145.

Proulx, S.R. 2001. Can behavioural constraints alter the stability of signalling equilibria? Proceedings of the Royal Society of London Series B-Biological Sciences 268: 2307–2313.

Putland, D. 2001. Has sexual selection been overlooked in the study of avian helping behaviour? Animal Behaviour 62: 811-814.

Reyer, H.U. 1984. Investment and relatedness - a cost-benefit analysis of breeding and helping in the pied kingfisher (Ceryle rudis). Animal Behaviour 32: 1163-1178.

Reyer, H.U. 1990. Pied kingfishers: ecological causes and reproductive consequences of cooperative breeding. In Cooperative Breeding in Birds: Long Term Studies of Ecology and Behaviour (Stacey, P.B. & W.D. Koenig, eds). Cambridge: Cambridge University Press. pp. 527-557.

Roberts, G. 1998. Competitive altruism: from reciprocity to the handicap principle. Proceedings of the Royal Society of London Series B-Biological Sciences 265: 427–431.

Számadó, S. 2011. The cost of honesty and the fallacy of the handicap principle. Animal Behaviour 81: 3–10.

Wright, J. 1997. Helping-at-the-nest in Arabian babblers: signalling social status or sensible investment in chicks? Animal Behaviour 54: 1439-1448.

Wright, J. 1998a. Helpers-at-the-nest have the same provisioning rule as parents: experimental evidence from play-backs of chick begging. Behavioral Ecology and Sociobiology 42: 423-429.

Wright, J. 1998b. Helping-at-the-nest and group size in the Arabian Babbler Turdoides squamiceps. Journal of Avian Biology 29: 105-112.

Wright, J. 1999. Altruism as a signal - Zahavi’s alternative to kin selection and reciprocity. Journal of Avian Biology 30: 108-115.

Wright, J., E. Berg, S.R. De Kort, V. Khazin & A.A. Maklakov. 2001a. Cooperative sentinel behaviour in the Arabian babbler. Animal Behaviour 62: 973-979.

Wright, J., A.A. Maklakov & V. Khazin. 2001b. State-dependent sentinels: an experimental study in the Arabian babbler. Proceedings of the Royal Society B-Biological Sciences 268: 821-826.

Wright, J., E. Berg, S.R. de Kort, V. Khazin & A.A. Maklakov. 2001c. Safe selfish sentinels in a cooperative bird. Journal of Animal Ecology 70: 1070-1079.

Wright, J. 2007. Cooperation theory meets cooperative breeding: exposing some ugly truths about social prestige, reciprocity and group augmentation. Behavioural Processes 76: 142-148.

Zahavi, A. 1975. Mate selection—selection for a handicap. Journal of Theoretical Biology 53: 205–214.

Zahavi, A. 1977. Reliability in communication systems and the evolution of altruism. In Evolutionary Ecology (Stonehouse, B. & C.M. Perrins, eds). London: Macmillan. pp. 253-259.

Zahavi, A. 1981. The lateral display of fishes—bluff or honesty in signaling? Behaviour Analysis Letters 1: 233–235.

Zahavi, A. 1990. Arabian babblers: the quest for social status in a cooperative breeder. In Cooperative Breeding in Birds: Long Term Studies of Ecology and Behaviour (Stacey, P.B. & W.D. Koenig, eds). Cambridge: Cambridge University Press. pp. 103-130.

Zahavi, A. 1991. On the definition of sexual selection, Fisher model, and the evolution of waste and of signals in general. Animal Behaviour 42: 501–503.

Zahavi, A. 1993. The fallacy of conventional signaling. Philosophical Transactions of the Royal Society of London Series B-Biological Sciences 340: 227–230.

Zahavi, A. 1995. Altruism as a handicap—the limitations of kin selection and reciprocity. Journal of Avian Biology 26: 1–3.

Zahavi, A. 2003. Indirect selection and individual selection in socioblology: my personal views on theories of social behaviour. Animal Behaviour 65: 859–863.

Zahavi, A. 2007. Sexual selection, signal selection and the handicap principle. In Reproductive Biology and Phylogeny of Birds (Jamieson, B.G.M., ed.). Enfield, USA: Science Publishers. pp. 143–160.

Zahavi, A. 2008. The handicap principle and signaling in collaborative systems. In Sociobiology of Communication: An Interdisciplinary Perspective (D’Ettorre, P. & D. P. Hughes, eds.). Oxford: Oxford Biology, pp.1–9.

Zahavi, A. & M. Perel. 2011. The information encoded by the sex steroid hormones testosterone and estrogen: A hypothesis. Journal of Theoretical Biology 280: 146–149.

Zahavi, A. & A. Zahavi. 1997. The Handicap Principle: A Missing Piece of Darwin’s Puzzle. Oxford: Oxford University Press.