EFTA02702876

Journal of lhearrtical Biology 399(2016) 103-116 1O4 aRt` SN ELSEVIER Contents lists available at ScienceDirect Journal of Theoretical Biology journal homepage: Conshlark Evolution of worker policing Jason W. Olejarz a, Benjamin Allen l'"', Carl yeller • Raghavendra Gadagkar -1. Martin A. Nowak a•d•g.s Program for twoltalonary Dynamks. Hanard University, Cambridge, MA 02(38. USA °Department of Mathematics. Emmanuel College, Bosron. MA 02115, USA `Center for Mathematical Sciences and Applicanons. Harvard Unhersky. Cambridge, MA 02138. USA Depaytmcnr of Organismic and Emlutionary Biology. Hamad University. Cambridge. MA 02178. USA • Centre for Ecological Sciences and Centre for Contemporary Studies. Indian Institute of Science. Bangalore 560 072. India 'Indian National Science Academy. New Delhi 110 002. India ▪ Deponent of Mathematics. Harvard University. Cambridge. MA OM& USA ARTICLE INFO Ankle history. Received 2 February 2015 Received in revised form 23 January 2016 Accepted 2 March 2016 Available online 11 March 2016 Keywords: Sociobiology Natural selection Evolutionary dynamics Modelsisimulations ABSTRACT Workers in insect societies are sometimes observed to kill male eggs of other workers, a phenomenon known as worker policing. We perform a mathematical analysis of the evolutionary dynamics of policing. We investigate the selective forces behind policing for both dominant and recessive mutations for dif- ferent numbers of matings of the queen. The traditional, relatedness-based argument suggests that policing evolves if the queen mates with more than two males, but does not evolve if the queen mates with a single male. We derive precise conditions for the invasion and stability of policing alleles. We And that the relatedness-based argument is not robust with respect to small changes in colony efficiency caused by policing. We also calculate evolutionarily singular strategies and determine when they are evolutionarily stable. We use a population genetics approach that applies to dominant or recessive mutations of any effect size. 2016 Elsevier Ltd. All rights reserved. 1. Introduction In populations with haplodiploid genetics, unfertilized female workers are capable of laying male eggs. Thus, in a haplodiploid colony, male eggs can in principle originate from the queen or from the workers. Worker policing is a phenomenon where female workers kill the male eggs of unmated female workers (Ratnieks. 1988; Ratnieks and Visscher. 1989; Ratnieks et al., 2006; Gadagkar. 2001; Wenseleers and Ratnieks, 20064. Worker policing is observed in many social insects, including ants, bees, and wasps. Yet the precise conditions for the evolution of worker policing are still unclear. Worker policing (Ratnieks. 1988; Ratnieks et al.. 2006; Gadagkar, 2001; Wenseleers and Ratnieks. 2006a) and worker sterility (Wilson. 1971; Hamilton. 1972; Olejarz et al., 2015) are two distinct phe- nomena that are widespread in the eusocial Hymenoptera. In addi- tion to worker policing, a subset of workers in a colony may forego their own reproductive potential to aid in raising their siblings. Prior relatedness-based arguments have suggested that queen monogamy is important for the evolution of a non-reproductive worker caste • Corresponding author. E-mail oddmss: martin novakPhaivard.edu (MA. Novak). latp:ndx.clorcag/1010I6Ultbi.20I6.03.091 0022-5193)&2016 Elsevier Ltd. All rights reserved. (Hughes et al.. 2008; Cornwallis et al.. 2010; Queller and Strassmann. 1998; Foster et al.. 2006; Rootnsma, 2007, 2009). In contrast, it is believed that polygamy—not monogamy—is important for the evo- lution of police workers. Several papers have studied the evolution of policing. Starr (1984) explores various topics in the reproductive biology and sociobiology of eusocial Hymenoptera. He defines promiscuity as 1 firl_ ill ). where n is the number of matings of each queen, and J; is the fractional contribution to daughters by the i-th male mate. He writes, regarding workers, that "They are on average less related to nephews than brothers whenever (promiscuity is greater than two) and should prefer that the queen lay all the male eggs. Workers would therefore be expected to interfere with each other's reproduction." Thus, Starr (1984) was the first to suggest that workers should raise their nephews (sons of other workers) if the queen mates once, but should only raise their brothers (sons of the queen) if the queen mates more than twice. Starr (1984) uses a relatedness-based argument. but he does not provide any calcu- lation of evolutionary dynamics in support of his argument; he uses neither population genetics nor inclusive fitness theory. In a book on honeybee ecology, Seeley (1985) also proposed, using a relatedness-based argument, that worker policing should occur in colonies with multiply mated queens, but that worker policing should be absent if queens are singly mated. EFTA_R1_02076880 EFTA02702876 104 1W. OhIan et al /journal of Theoretkal Bletrov 399 (2016)103-116 Woyciechowski and Lomnicki (1987) perform a calculation based on population genetics and conclude that workers should raise their nephews (sons of other workers) if the queen mates once, but should only raise their brothers (sons of the queen) if the queen mates more than twice—the case of double mating is neu- tral with respect to preference. From this result, they claim that, under multiple mating of the queen, natural selection should favor non-reproductive workers. Woyciechowski and Lomnicki (1987) consider both dominant and recessive alleles affecting worker behavior, but they do not consider colony efficiency effects. Ratnieks (1988, considers the invasion of a dominant allele for policing. Using population genetics, he arrives at essentially the same conclusion as Woyciechowski and Lomnicki (1987): In the absence of efficiency effects, policing evolves with triple mating but not with single mating. But Ramieks also considers colony efficiency effects, focusing mainly on the case where policing improves colony efficiency. Since policing occurs alongside other maintenance tasks (such as cleaning of cells. removal of patho- gens, incubation of brood), and since eating worker-laid eggs might allow workers to recycle some of the energy lost from laying eggs. Ramieks supposes that policing improves colony efficiency. He finds that worker policing with singly mated queens may evolve if policing improves colony reproductive efficiency. He also finds that worker policing with triply mated queens may not evolve if policing reduces colony reproductive efficiency, but he considered this case to be unlikely on empirical grounds. Ratnieks does not study recessive policing alleles. He also does not calculate evolutionary stability conditions. Both papers (Woyciechowski and Lomnicki. 1987; Ratnieks, 1988) offer calculations based on population genetics without mentioning or calculating inclusive fitness. These early studies (Starr. 1984; Seeley, 1985; Woyciechowski and Lomnicki. 1987; Ratnieks. 1988) were instrumental in establishing the field of worker policing. Testing theoretical predictions on the evolution of worker policing in the field or in the lab is difficult Due to the complex- ities inherent in insect sociality, published empirical results are not always easy to interpret. While, so far, winter policing has been found in all species with multiple mating that have been studied, it has also been found in about 20% of species with singly mated queens (I lammond and Keller. 2004; Wenseleers and Ratnieks. 2006b; Bonckaert et al.. 2008). Herein lies the difficulty: When worker policing is found in multiply mated species and found to be absent in singly mated species, this is taken as evi- dence supporting the relatedness argument, and when worker policing is found in singly mated species, it is explained away as not being evidence against the theory, but as having evolved for other reasons (such as colony efficiency). See, for example, the following quotation by Bonckaert et al. (2008): 'Nevertheless. our results are important in that they show that V. germanica forms no exception to the rule that worker reproduction should be effec- tively policed in a species where queens mate multiple times (Ratnieks, 1988). Indeed, any exception to this pattern would be a much bigger challenge to the theory than the occurrence of worker policing in species with single mating, which can be readily explained (Ratnieks, 1988; Foster and Ratnieks. 2001b)." This is precisely why a careful simultaneous consideration of relatedness, male parentage, and colony efficiency is important for understanding worker policing. We do not aim to provide an exhaustive catalog of all species in which worker policing has been studied. We merely cite some specific examples to add context Policing is rampant in colonies of the honeybee (Ratnieks and Visscher. 1989), the wasp Vesputa vutgaris (Foster and Ratnieks, 2001c), and the wasp Vespula ger- manica (Bonckaert et al., 2008), which are all multiply mated. (As mentioned above, worker policing has been found in all of the studied species to date that are multiply mated.) Worker removal of worker-laid eggs is much less prevalent in colonies of the bumblebee (Velthuis et al., 2002), the stingless bee, (Peters et al.. 1999), and the wasp, Vespula rata (Wenseleers et al.. 2005), which are predominantly singly mated. (As mentioned above, worker policing has been found only in about 20% of the studied species to date that are singly mated.) There are some studies based on observational evidence that find policing in singly mated species; examples of species with single mating and worker policing are Vespa crobro (Foster et al.. 2002), Camponotusjlortdanus (Endler et al., 2004), Aphaenogaster smythiesi (Wenseleers and Ratnieks, 200bh). and DitIC0MM0 (Wenseleers and Ratnieks, 20061), Interspecies comparisons are somewhat problematic, because even though phylogeny can be controlled for, there are many (known and unknown) ways in which species differ in addition to mating frequency that may also affect the absence or presence of worker policing. Furthermore. many empirical studies are based on genetic analyses of male parentage. (Though studies of some species are based on actual observational evidence: see. e.g., Wenseleers and Ratnieks, 2006b.) Regarding species for which the study of policing is based on genetic analyses, policing is often inferred if males are found to originate predominantly from the queen. But such an inference, in cases where it is made, pre- supposes that workers actively try to lay male eggs in the first place. It is therefore not clear how reliably genetic investigations can measure policing. The small number of attempts at measuring the prevalence of worker policing in intraspecific experiments have also returned conflicting results. Foster and Ratnieks (2000) report that facul- tative worker policing in the saxon wasp. Dolichovespula saxonica. is more common in colonies headed by multiply mated queens. But their sample size is only nine colonies. The phenomenon was reinvestigated by Ronckaert et al. (2011) who report no evidence of facultative worker policing depending on queen mating fre- quencies, and argue that the previous result may have been flawed or that there were interpopulational variations. Many empirical studies have emphasized that factors besides intracolony relatedness—including the effects of policing on a colony's rate of production of offspring—may play a role in explaining evolution of worker policing (Foster and Ratnieks 2001a,c: Hartmann et al_ 2003; Hammond and Keller. 2004; Wenseleers and Ratnieks. 2006b; Helantera and Sundstrom. 2007; Khila and Abouheif. 2008; Zanette et al. 2012). Yet reliable pub- lished data on the effect that policing has on colony reproductive efficiency are often hard to find. (For some exceptions, see Wenseleers et al.. 2013 and references therein.) In this paper, we derive precise conditions for the evolutionary invasion and evolutionary stability of police alleles. We consider any number of matings, changes in the proportion of queen- derived males, changes in colony efficiency, and both dominant and recessive mutations that affect the intensity of policing, Our paper is based on an analysis of evolutionary dynamics and population genetics of haplodiploid species (Nowak et al., 2010; Olejarz et al., 2015). It does not use inclusive fitness theory. Spe- cifically, we adapt the mathematical approach that was developed by Olejarz in al. (2015) for the evolution of non-reproductive workers. We derive evolutionary invasion and stability conditions for police alleles. Mathematical details are given in Appendix A. In Section 2, we present the basic model and state the general result for any number of matings for dominant policing alleles. In Sections 3-5, we specifically discuss single, double. and triple mating for dominant policing alleles. We take dominance of the policing allele to be the more realistic possibility because the policing phenotype is a gained function. Nonetheless. for com- pleteness, we give the general result for recessive policing alleles in Section 6. In Section 7, we discuss how the shape of the EFTA_R1_02076881 EFTA02702877 Olefore et at /Journal of Theorrtkol Biology 399 (20:6) I03-II6 105 efficiency function determines whether or not policing is more likely to evolve for single or multiple matings. In Section 8. we analyze our results for the case where the phenotypic mutation induced by the mutant allele is weak (or, equivalently in our formalism, the case of weak penetrance). In this setting, the quantity of interest is the intensity of policing. We locate the evolutionarily singular strategies. These are the values of intensity of policing for which mutant workers with slightly different policing behavior are, to first order in the mutant phenotype. neither advantageous nor disadvantageous. We then determine if a singular strategy is an evolutionarily stable strategy (ESS). In Section 9. we discuss the relationship between policing and inclusive fitness theory. together with the limitations of the relatedness-based argument. Section 10 concludes. 2. The model We investigate worker policing in insect colonies with haplo- diploid genetics. Each queen mates n times. We derive conditions under which a mutation that effects worker policing can spread in a population. We make the simplifying assumption, as do Wowiechowski and Loninicki (1987) and Rat nicks (1988). that the colony's sex ratio is not affected by the intensity of worker policing. First we consider the case of a dominant mutant allele. Because the policing allele confers a gain of function on its bearer, the assumption that it is dominant is reasonable. There are two types of males. A and a. There are three types of females, AA. Aa, and aa. If the mutant allele is dominant then Aa and aa workers kill the male eggs of other workers, while M workers do not. (Alter- natively, M workers police with intensity ZAA, while Aa and ao workers police with intensity Zaa = =Zm+w. We consider this case in Section 8.) For n matings, there are 3m + 1) types of mated queens. We use the notation AA, in; Ac. in; and art m to denote the genome of the queen and the number, m. of her matings that were with mutant males. a. The parameter m can assume values 0.1....,n. A schematic of the possible mating events is shown in Fig. 1(a). There are three types of females, AA. Aa. and aft and there are n+1 possible combinations of males that each queen can mate with. (For example, a queen that mates three times (n=3) can mate with three type A males. two type A males and one type a male, one type A male and two type a males, or three type a males.) Fig. I (b) shows the different colony types and the offspring of each type of colony when each queen is singly mated. Fig. I (c) shows the different colony types and the offspring of each type of colony when each queen mates n times. The invasion of the mutant allele only depends on a subset of colony types. The cal- culations of invasion conditions are presented in detail in Appendix A. 2.1. Fraction of male offspring produced by the queen pa represents the fraction of males that are queen-derived if the fraction of police workers is z. (This quantity was already employed by Ratnieks. 1988.) The parameter z can vary between 0 and 1. For z=0. there are no police workers in the colony, and for z= 1, all workers in the colony are policing. We expect that pa is an increasing function of z. Increasing the fraction of police workers increases the fraction of surviving male eggs that come from the queen (Fig. 2). a vimin Queens Males PA + (n —m) A +m a Aa aa + (n —m) A +m a + (n —m ) A+ma p Fertiliad Queens AA,m Aa.m aafri San Cbiab ninisrb• Tem Clusll O•WIIPO Ors•if• INI•oss• Os" Ns Maw' Moo AAA Ms ••• AAA rs A. • A Aa A•••• M•a• A• •• A • • • A•• • C ra Come likes a Usua •• • SA0a Awl* A •.0 Oman timain• Porn% ClougAlm• 0.••••• scos- C•i•Ase Sam -ANAA•mAs A (2.•••1••••• Awn \10-1•0•A•nAs•maa A•• 1.0 A • In•2al• ran la-m)M•air In-elA•01•••• nip I (a) The possible fluting events with haplodiploid genetics are shown. Each queen mates with n malts. m denotes the number of times that a queen mates with mutant type a males and can take values between 0 and n. Thus. there are 3Ot I I) types of colonies. (b) If each queen mates with only a single male, then there are six types of colonies. The female and male offspring (right three columns) of each colony (leftmost column) are shown. For example. M, I colonies arise from a type M female fluting with a single mutant type a male. AA. I queens produce female offspring of type Aa and male offspring of type A. 50% of the offspring of workers in AA. I colonies ate of type A. while the remaining 50% of the offspring of workers in M. I colonies are of type o. (c) The female and male offspring fright three columns) of each colony (leftmost column) when each queen mates n times are shown. a 5 eiei, 0 0.25 0_ 0.25 0.50 0.75 Fraction of police workers, Ng. 2. The queen's production of male eggs. pt. increases with the fraction of waiters that are policing. Z. This is intuitive, since having a larger worker police force means that a greater amount of worker-laid eggs can be eaten or removed. Three possibilities for a monotonically increasing function p, are shown. Possible Forms for the Function p, 0.75 0.50 22 Colony efficiency as a function of policing 1 r, represents the rate at which a colony produces offspring (virgin queens and males) if the fraction of police workers is z. (This quantity was also employed by Ratnieks. 1988.) Without loss of generality, we can set rc, = I. For a given mutation that affects the intensity of policing, and for a given biological setting, the efficiency function ra may take any one of a variety of forms (Fig. 3). Colony efficiency depends on interactions among police work- ers and other colony members. It also depends on the interactions of colonies and their environment. There are some obvious nega- tive effects that policing can have on colony efficiency. By the act EFTA_R1_02076882 EFTA02702878 1013 1.3 1.2 j 1.1 73 I 1 I 0.9 0.8 ale/an et at /journal of Theoretical Biology 399 (2016) 103-116 Possible Forms for the Function r, 0 0.25 0.50 0.75 Fraction of police workers, z Fig. 3. The functional dependence of colony efficiency. r,, on the fraction of workers that are policing. z. may take any one of many possibilities. of killing eggs, police workers are directly diminishing the number of potential offspring. In the process of identifying and killing nephews, police workers may also be expending energy that could otherwise be spent on important colony maintenance tasks (Cole, 1986; Naeger et al.. 2013). Policing can also be costly if there are recognitional mistakes. i.e.. queen-laid eggs may accidentally be removed by workers. Recognitional errors could result in modifications to the sex ratio. which is an important extension of our model but is beyond the scope of this paper. We can also identify positive effects that policing may have on colony efficiency. It has been hypothesized that the eggs which are killed by police workers may be less viable than other male eggs (Velthuts et al., 2002: Pirk et al.. 1999: Gadagkar. 2004; Nonacs, 2006), although this possibility has been disputed (Beekman and Oldmyd. 2005; Helantera et al.. 2006; Zanetre et al.. 2012). If less- viable worker-laid eggs are competing with more-viable queen-laid male eggs, then policing may contribute positively to overall colony efficiency. Moreover, policing decreases the incentive for workers to expend their energy laying eggs in the first place (Foster and Ratnieks, 2001a; Wenseleers et al.. 2004a,b: Wenseleers and Ratnieks. 2006a). which could be another positive influence on colony efficiency. (However, the decrease in incentive for workers to reproduce due to policing would only arise on a short time scale if there is a facultative response to policing which is unlikely.) As another speculative possibility: Could it be that worker egg- laying and subsequent policing acts as a form of redistribution within the colony? That is. suppose that it is better for colony efficiency to have many average-condition workers than to have some in poor condition and some in good condition. Suppose further, as seems realistic. that good-condition workers are more likely to lay eggs (which are high in nutritional content, of course). If the average police worker is of condition below the average egg- laying worker, then worker egg-laying and policing serves to redistribute condition among the workers, improving overall col- ony efficiency. The special case, where policing has no effect on colony effi- ciency and which has informed the conventional wisdom, is ungeneric, because policing certainly has energetic consequences for the colony that cannot be expected to balance out completely. An early theoretical investigation of colony efficiency effects regarding invasion of dominant mutations that effect worker policing was performed by Ratnieks (1988). Although monotonically increasing or monotonically decreas- ing functions r, are the simplest possibilities, these cases are not exhaustive. For example, a small or moderate amount of policing may be expected to improve colony efficiency. However, the pre- cise number of police workers that are needed to effectively police the entire worker population is unclear. It is possible that a frac- tion z < 1 of police workers can effectively police the entire population, and adding additional police workers beyond a certain point could result in wasted energy, inefficient use of colony resources, additional recognitional errors. etc. These effects may correspond to colony efficiency r, reaching a maximum value for some 0 < z <1. As another possibility, suppose that police workers, when their number is rare, directly decrease colony efficiency by the act of killing male eggs. It is possible that for some z c 1, police workers are sufficiently abundant that their presence can be detected by other workers. Assuming the possibility of some type of facultative response, the potentially reproductive workers may behaviorally adapt by reducing their propensity to lay male eggs, instead directing their energy at raising the queen's offspring. In this scenario, colony efficiency r, may reach a minimum value for some 0<z<1. 23. Main results for dominant police alleles We derive the following main results for dominant police alleles. If the queen mates with n males, then the a allele for policing can invade an A resident population provided the fol- lowing "evolutionary invasion ( condition" holds: pun +pia (rim\ ma) > 2_ I'l_ _„ .1 trim\ 2 ro ro 1'0 l ring ro (1) When considering only one mutation. ro can be set as 1 without loss of generality. Why are the four parameters. rim. ruz, p", and p12, sufficient to quantify the condition for invasion of the mutant allele, a? Since we consider invasion of a. the frequency of the mutant allele is low. Therefore, almost all colonies are of type AA 0. which means a wild-type queen. M. has mated with n wild- type males, A. and 0 mutant males, a. In addition, the colonies Aa, 0 and M. I are relevant. These are all colony types that include exactly one mutant allele. Colony types that include more than one mutant allele (such as Aa.1 or AA 2) are too rare to contribute to the invasion dynamics. For an Aa.O colony, half of all workers are policing and therefore the parameters r1j2 and p1,2 occur in Eq. (1). For an AA.1 colony, 1/n of all workers are policing, which explains the occurrence of rim and p,,,, in Eq. (1). Next, we ask the convene question: What happens if a popu- lation in which all workers are policing is perturbed by the introduction of a rare mutant allele that prevents workers from policing? If the a allele for worker policing is fully dominant, and if colony efficiency is affected by policing then a resident policing population is stable against invasion by non-police workers if the following "evolutionary stability condition" holds: ri (2 + 0)(2 + pi) +pa„_ ivamtn-2) > (2) ran- mom 2(2+n+npi ) What is the intuition behind the occurrence of the four para- meters, r1. ran_ wop pi. and pan _ warn? The condition applies to a population in which all workers are initially policing. Note that. because the allele, a. for policing is fully dominant in our treat- ment. non-policing behavior arises if at least two mutant A alleles for non-policing are present in the genome of the colony. which is the combination of the queen's genome and the sperm she has stored. To study the invasion of a non-policing mutant allele. we must consider all colony types that have 0,1, or 2 mutant A alleles: these are Oa. n; aa.n —1: Ao.n: aa. n —2; Ao.n-1; and M. n. The colonies aa,n: aa.n —1: Aa.n: aa.n —2: and Mae do not contain non-police workers: the efficiency of those colonies is r,, and the fraction of male eggs that originate from the queen in those colonies is pi. Both of these parameters occur in Eq. (2:. Colonies of type Aa.n - 1 produce a fraction of 1/(2r) non-police workers. EFTA_R1_02076883 EFTA02702879 Olejarz et at /Journal of Theorrtkol Biology 399 (2016)103-N6 107 a 100 a 101 10 Single mating. n=1 — r •052 - rp1.061 11s1.0130 — r et• 069 10 Time (104) 15 Fig. 4. Numerical simulations of the evolutionary dynamics of wetter policing team the condition given by Eq. ' I :. The policing allele is dominant. For numerically probing invasion we use the initial condition XA•o= 1-10- 3 and XAA 1=10 3. We set ro= I without loss of generality. Other parameters are: (a)Pi7 =0.75. pl =0.9. and (b) Ave 0.6, r — 1005. and ri -1.01. • C— C— Police allele ca ' ade and is evolutionarily stable ca ade. but is not stable to not vale but s stable c not d nil is not stable 0 1 Frequency of police allele Fig. 5. There are four possibilities for the dynamical behavior in the proximity of two pure equilibria. a n = 1 mating 1.2 .-- co e.= 11 1 C O O 0.9 p1,2 = 0.75, p, = 1 Stably Does Not \ Invade Invades \ Unstable / Does Not Invade Bitable Stable 0.9 1 1.1 1 2 Colony efficiency, r v2 b n = 1 mating p,,, = 0.99. p, = 1 1.2 Colony efficiency, r1 1.1 0.9 - 10.8 Invades Unstable Does Not Invade Unstable 0.9 1 1.1 Colony efficiency, r1,2 1.2 Fig. S. If queens are singly mated (n-1). then a plot of I.' versus r1,2 dearly shows all four possibilities for the behavior around the two pure equilibria. For (a), we set pi 0.75 and pi 1. For (b). we set pi = 0.99 and p,-1. which explains the occurrence of ri2,102a, and pa,,_102„, in Eq. (2). Numerical simulations of the evolutionary dynamics with a dominant police allele are shown in Fig. 4. Generally, four scenarios regarding the two pure equilibria are possible: Policing may not be able to invade and be unstable, policing may not be able to invade but be stable, policing may be able to invade but be unstable, or policing may be able to invade and be stable. The possibilities are shown in Fig. 5. In the cases where policing cannot invade but is stable, or where policing can invade but is unstable. Brouwer's fixed-point theorem guarantees the existence of at least one mixed equilibrium. In the case where policing can invade but is unstable, police and non-police workers will coexist indefinitely. We will now discuss the implications of our results for parti- cular numbers of matings. E F TA_R 1_02076884 EFTA02702880 lea J.W. Otejaa ef al / Journal of Theo:ilk& Swim 399 (2016)103-116 a The police allele invades and is stable. p,2 0.75. r,? 1.0344, re1.0787 1 1 0.8 1 0.6 0 • 0.2 LL 1 0.8 2 3 Time (10') 4 The police and non-police alleles are bistable. p,a 40.75. rflu1.0244. r,a1.0667 0.6 - OA 5 10 15 Time (103) 20 b a -8 I d The non.polico allele invades and is stable. pin=0.75. p.=1. rig=1.0344. r,=1.0567 0.8 0.8 - OA 0.2 - 0 1 0.8 O • 0.8 a 15 0.4 5 02 LL 00 84 88 88 Thin (10') The police and non-poise alleles meat pws0.75. pint ri4e1 .0444. ?ter 1 4 6 8 10 Time (10') Flg. 7. Nurnencal simulations of the evolutionary dynamics of worker policing that show the four behaviors in hg (,(a). The policing allele is dominant For each of the four panels. we use the initial conditions: (a) XAto I -10 l and gm - 10 1:(b)X,ai - I -10 s and r eso - 10 3: 01 %Am) 0.02 and Xµl— 098 (lower curve). and Xmo = 0 01 and XAA l =0.99 (upper cum): (d)44.0 =1-10-2 and XAAI =10-2 (lower curve). and )(mi I —10-l and 4, 10-2 (upper curve). We set ro • I without loss of sawrality. 1.003 4.." 1.002 1.001 0.999 0.998 n = 1 mating Petty Wades bilis unstable Pollee Insides aid et HAW — Mktg der not Merle and a unstable — eons do a Minna, but e ;We 0 1/2 1 Fraction of police workers. Fig. 8. Possible r, efficiency curves for n=1 mating which demonstrate different behaviors. For this plot, we set pre =099 and m 41. Here. each curve has the functional form r1 - 1 +ea +922. For exam*. we can have: (blue) policing Invades but is unstable, o-0.003. )l -0.0004: (green) policing invades and is stable. (7=0.0026. 4=0: (red) policing does not invade and is unstable. 0=0.0024. p=0: (black) policing does not Invade but is stable. 0=0.002. 4=00004. (For Inter- pretation of the references to color in this figure caption, the reader is referred to the web version of this paper.) 3. Single mating For single mating. n= I. the invasion condition for a dominant police allele is 2(2 —r1/2) > 2(1 —PI ) 441 +PI,.2)rla (Recall that ro a 1.) 13) The stability condition for a dominant police allele is ri > 6—pia+3p,I-112 (4) 6+2p, Evolution of policing is highly sensitive to changes in colony effi- ciency. For example, let us consider p, ,2 = 0.99 and pi = 1. This means that if half of all workers police then 99% of all males come from the queen. If all workers police then all males come from the queen. In this case, efficiency values such as r 1,2 = 1.001 and r, = 1.0031 lead to the evolution of policing. In principle, arbitrarily small increases in colony efficiency can lead to the evolution of policing for single mating. A plot of r, versus rif2 for singly mated queens (Fig. 6) illus- trates the rich behavior highlighted in Fig. 5. Numerical simula- tions of the evolutionary dynamics are shown in Fig. 7. Another intriguing feature is that increases in colony efficiency due to policing do not necessarily result in a higher frequency of police workers at equilibrium Fig. S illustrates this phenomenon. Four possibilities for the efficiency function r., are shown. Notice that the r2 curve which results in coexistence of police workers and non- police workers (blue, top) is strictly greater than the r2 curve which results in all workers policing (green, second from top). How can increased efficiency due to policing possibly result in policing being less abundant at equilibrium? If a mutation for non-policing behavior is introduced into a resident policing population, then the evolu- tionary success of the non-policing mutation depends on the success of Ao. 0 colonies relative to aa. 1. aa. 0. Aa. 1, and AA, 1 colonies. Au. 0 colonies have an efficiency parameter r,,2. while the other four relevant colonies each have an efficiency parameter ri. Thus, if via is too large relative to r,, then the non-police allele can invade a resi- dent policing population, and there is coexistence. Also notice that the r, curve which results in bistability of police workers and non-police workers (black, bottom) is strictly less than EFTA_R1_02076885 EFTA02702881 J.W. Olefarz et at /Journal of Theowrkal Biology 399 (2016)103-H6 109 n = 2 matings 1.003 - noising irniszles WI Is onstobte .7 1.002 — Poland invades and is slab* painng does nor ',wady end is unstable Poking don not made bulls**. •_ 0 a) 1.001 oror 0 (.) 0.999 (-> 0.998 a 1/2 3/4 1 Fraction of police workers,: Fig. 9. Possible r, efficiency curves for n=2 marines which demonstrate different behaviors Here, each curve has the functional form r, • fir 2. For example. we can have: (blue) policing invades but is unstable. a= 0.0005. D=-0.0004: (green) policing invades and is stable. 0=0.000l. /3=0: (red) policing does not invade and is unstable. a= -0.000). D=O: (black) policing does not invade but is stable. a= -0.0005. rre0.0004. (For interpretation of the references to color in this figure caption. the reader is referred to the web version of this paper.) the r, curve which results in policing being dominated by non-policing (red, second from bottom). This phenomenon arises in a similar way: if r, .2 is too small relative tor,, then the non-police allele cannot invade a resident policing population, and there is bistability. 4. Double mating For double mating. n=2. the Invasion condition fora dominant police allele is given by r1,2 > 1 (5) Thus, policing can invade if there is an infinitesimal increase in colony efficiency when half of all workers police. Policing cannot invade if there is an infinitesimal decrease in colony efficiency when half of all workers police. The stability condition for policing is given by r, > r314 (61 Therefore, the policing allele is stable if the colony efficiency is greater for z=1 (when all workers police) than for z = 3/4 (when only three quarters of the workers police). Four possible efficiency curves r, and the corresponding behavior of the police allele are shown in Fig. 9. 5. Triple mating For triple mating. n=3. the invasion condition for a dominant police allele is given by 4-2(1 —pliorla rIR > 2 +(pin +Pnoria The stability condition for policing is given by 101-p5/6-F5pi rl > r516 10+6pi (7) (8) As a numerical example, let us consider p1„.3 = 0.98 and pia =0.99.11x=1/3 of workers police. then 98% of males come from the queen. If z = 1/2 of workers police, then 99% of males come from the queen. In this case, policing cannot invade if ri,3 =0.9990 and ria =0.9979. In principle, arbitrarily small reduc- tions in colony efficiency can prevent evolution of policing for triple mating. 1.003 1.002 1.001 1 0.999 0.998 n = 3 mations Peking inv.. WI isunstable — Poland }made* and is slat. Poicing does not Mrs. and n unstable - Pohang does not info. toil is stable a 1/3 1/2 (2/3) 5/6 1 Fraction of police workers,: Fts.10. Possible r, efficiency curves forn-3 matings which demonstrate different behaviors. For this plot we set p,r, =0.986. pe, = 0.99. Ps,. = 0996. and pi = 1. Here, each curie has the functional form r, = I -,a-par. For example. we can have: (blue) policing invades but is unstable. a= - 0.0006..d -0.0006: (green) policing invades and is stable, a--0.0012. D-0: (red) policing does not invade and is unstable. a= -00015. 0=0: (black) policing does not invade but is stable. • 0.0021. A-now& Note that the value r2,1 affects the population dynamics but does not appear in the conditions for invasion and stability of the police allele. hence the parentheses on the horizontal axis. (For interpretation of the references to color in this figure caption. the reader is referred to the web version of this paper.) Just as for single mating, we observe the intriguing feature that increases in colony efficiency due to policing do not necessarily result in a higher frequency of police workers at equilibrium. Fig. 10 illus- trates this phenomenon. Four possibilities for the efficiency function I-, are shown. Notice that the r, curve which results in coexistence of police workers and non-police workers (blue, top) is strictly greater than the r, curve which results in all workers policing (green, second from top). Also notice that the r, curve which results in bistability of police workers and non-police workers (black, bottom) is strictly less than the r; curve which results in policing being dominated by non- policing (red, second from bottom). 6. Recessive police allele We have also derived the conditions for the emergence and evo- lutionary stability of worker policing if the police allele is fully reces- sive. In this case, AA and Aa workers are phenotypically identical and do not police, while aa workers do police. (Alternatively. AA and M workers police with intensity ZA4=Zed. while no workers police with intensity Ze, =4,4+w = 4„ + w. We consider this case in Section 8.) 6.1. Emergence of worker policing The invasion condition for a recessive police allele, a, is given by rya., > 2(2+n +npo) (9) r0 (2 + n)(2+po)+Tho2,0(n — 2) Note that Eq. (9) for invasion of a recessive police allele has the same mathematical form as Eq. (2) for evolutionary stability of a dominant police allele. Starting from Eq.:2'n making the substitution z—. 1—z, and reversing the inequality, we recover Eq. (9). The intuition behind this correspondence is described in Appendix A. 6.2. Stability of worker policing A recessive police allele, a. is evolutionarily stable if \ 12( \ _11 _0 nit\ _ nj j )kna) 2 (10) Note that Eq. (10) for evolutionary stability of a recessive police allele has the same mathematical form as Eq. (I) for invasion of a E FTA_R1_02076886 EFTA02702882 110 J.W. OkJarz er at /Journal of Theorrtkal Norm 399(2016)103-116 0 4 8 12 Time (105) Time (105) Rd. It. Numerical simulations of the evolutionary dynamics of worker policing confirm the condition given by Eq. . The policing allele is recessive. For numerically probing invasion. we use the initial condition Xmo - I -10-2 and Xmo t0-2. We set rot - I without loss of generality. Other parameters ale: (a) p0-0.6. p,,, 0 and (b) Po 036.PIA2 = 0.9. ,,rot 004, and rl = 1.012. Under what conditions does worker policing invade? n = 1 mating n= 2 mat%igs n= 3 mating' decreases monotonically Increases monotonically reaches a maximum for some 0 < z <1 reaches a minimum for some 0<z<1 NO NO NO NO NO Fig 12. Depending on the functional loon of colony efficiency. r1, on the fraction of police workers. z. policing alleles may or may not invade for single, double. or triple mating. Various possibilities of , arc shown. The outcomes hold for both dominant and recessive police alleles. If r, is constant. then policing does not invade for single mating. is neutral for double mating, and invades for triple mating. If r: decreases monotonically. then policing does not invade or invades only (or triple mating. If r: increases monotonically. then policing either invades only for double and triple mating or for single. double. and triple mating. If r, reaches a maximum at an intermediate value 0 <z < I. then policing does not invade or may invade (or triple mating only, for double and triple mating, or for single. double, and triple mating. If r„ reaches a minimum at an intermediate value 0 <z < I. then any pattern is possible. dominant police allele. Starting from Eq. (1), making the sub- stitution z I — z. and reversing the inequality, we recover Eq. ( 10). Again, the intuition behind this correspondence is described in Appendix A. Numerical simulations of the evolutionary dynamics with a recessive police allele are shown in fig. II. 7. Shape of the efficiency function. r, The shape of the efficiency function, G. determines whether policing is more likely to evolve for single mating or multiple matings. Recall that r, is the colony efficiency (defined as the rate of generation of reproductives) if a fraction, z. of all workers per- form policing. The variable z can assume values between 0 and 1. If no workers police, z= O. then the colony efficiency is at baseline, which we set to one; therefore, we have ro= I. Policing can in principle increase or decrease colony efficiency (Fig. 12). We have the following results regarding the invasion and sta- bility of police workers. We discuss single (n-1). double (n=2), and triple (n-3) mating. All results apply to both dominant and recessive police alleles. They can be instantiated with arbitrarily small changes in colony efficiency. 7.1. Evolutionary invasion of policing (i) If r, is strictly constant (which is ungeneric), then policing does not invade for single mating, is neutral for double mating, and does invade for triple mating. (ii) If r, is monotonically decreasing, then policing either invades not at all or only for triple mating. (iii) If r, is monotonically increasing, then policing either invades for single, double, and triple mating or only for double and triple mating. (iv) If r, reaches an intermediate maximum (which means colony efficiency is highest for an intermediate fraction of police E F TA_R 1_02076887 EFTA02702883 J.W. Olefare et at /Journal of Theorrtkol Biology 399 (2016) 103-116 111 a 1.012 1.010 1.008 1.006 1.004 1 002 0.998 Poking invades for n=1 but not for net b Policing does not invade but is stable (for ne1 and ne2) 1 - • 0.999 • • 0.998 • • • 0.997 • • 0 12 3(4 1 0.996 0 12 314 Fraction of police workers.: C Policing invades but is unstable ((or n=1 and n=2) 1.002 C 1.001 • • • • Fraction of police workers.: d Poking does not invade but is stable (for ne1) Poking invades but is unstable (for ne2) 1.003 4 I 1.002 1.001 1 • • • • 0 1/2 3/4 1 0 1/2 314 1 Fraction of poke workers.: Fraction of police workers.: Fig. 13. Non-monotonic efficiency functions can lead to rich and counterintuitive behavior. We consider invasion and stability of a dominant police allele for single (n I) and double (n-2) mating. The baseline colony efficiency without policing is re - I. Mice other values must be specified: ri and r,. Moreover. we need to specify two values for how the presence of police workers affects the fraction of malt offspring coming from the queen; we choose Viz —0.99 and Pi — 1. A variety of behaviors can be realized by a very small variation in colony efficiency. (a) Policing invades for single mating but not for double mating. (b) Policing does not invade but is stable for single and double mating. (c) Policing invades but is unstable for single and double mating. (d) Policing does not invade but is stable for single mating. while policing invades but Is unstable for double mating. workers). then policing can invade for n= 1.2.3 or n=2.3 or n=3 or not at all. (v) If rz reaches an intermediate minimum (which means colony efficiency is lowest for an intermediate fraction of police workers), then policing can invade with any pattern of mat- ings. For example, it is possible that policing invades only for single mating but neither for double nor triple mating. Or it invades for single and double mating but not for triple mating. 7.2. Evolutionary stability of policing (i) If rz is constant. then policing is unstable for single mating, is neutral for double mating, and is stable for triple mating. (ii) If r, is monotonically decreasing, then policing is unstable for single and double mating. For triple mating it can be stable or unstable. (iii) If r, is monotonically increasing, then policing either is always stable or is stable only for double and triple mating. (iv) If r, reaches an intermediate maximum, then policing can be stable for any pattern of matings. For example, policing can be stable for single mating but neither for double nor triple mating. (v) If r; reaches an intermediate minimum, then policing can be stable for n = 1.2.3 or n=2.3 or n=3 or not at all. 7.3. Examples for single and double mating Fig. 13 gives some interesting examples for how non- monotonic efficiency functions can influence the evolution of policing for single (n=1) and double (n=2) mating. In order to discuss the invasion and stability of a dominant police allele for single and double mating, we need to specify efficiency at three discrete values for the fraction of police workers present in a colony: r112. r314. and rt. Note that re, = 1 is the baseline. Moreover. we need to specify the fraction of male offspring coming from the queen at two values: P1,2 and pi. For all examples in Fig. 13, we assume pi/2 =0.99 and PI = 1. We show four cases: (a) policing invades for single mating but not for double mating; (b) for both single and double mating, policing does not invade but is stable: (c) for both single and double mating. policing invades but is unstable (leading to coexistence of policing and non-policing alleles): (d) policing does not invade but is stable for single mat- ing: policing invades but is unstable for double mating. These cases demonstrate the rich behavior of the system. which goes beyond the simple view that multiple matings are always favor- able for the evolution of policing. 8. Gradual evolution of worker policing Our main calculation applies to mutations of any effect size. In this section. we calculate the limit of incremental mutation (small mutational effect size). Our calculations in this section are remi- niscent of adaptive dynamics (Nowak and Sigmund, 1990; Hof- bauer and Sigmund. 1990; 0ieckmann and Law. 1996; Metz et al.. 1996; Geritz et al., 1998). which is usually formulated for asexual and haploid models. The analysis in this section applies both to the case of small phenotypic effect and to the case of weak penetrance. Mathematically, we consider the evolutionary dynamics of poli- cing if the phenotypic mutations induced by the a allele are small. If an allele affecting intensity of policing is dominant, then it is intui- tive to think of wild-type workers as policing with intensity ZA.R. E F TA_R 1_02076888 EFTA02702884 111 Oidon et al /journal of Theoretical Biology 399(2016)103-116 while mutant workers police with intensity 4. =4, =ZAA 1-w. If an allele affecting intensity of policing is recessive, then it is intuitive to think of wild-type workers as policing with intensity ZA4=ZA„, while mutant workers police with intensity Zo, = ZAA +V/ = Z,1, +w. In the limit of incremental mutation, the fraction. p. of queen- derived males and the colony efficiency, r, become functions of the average intensity of policing in the colony, which is Z+ wz. where z is the fraction of mutant workers in the colony. We have PIZ +vvz). Ra+P( )wz+! Ribv2z2 +O(w3) r,-.R(Z+wz)= R(21+ ralwz +I R. (Znv2z2 +O(tv3) (11) We have made the substitutions p,-•P(Z+ wz) and rz -.R(Z+wZ). and (11) gives the Taylor expansions of these quantities in terms of their first and second derivatives at intensity Z. (For conciseness, we will often omit the argument Z from the functions P and R and their derivatives.) Here, wl<1. so that workers with the phenotype corresponding to the mutant allele only have an incremental effect on colony dynamics. Thus, the expansions (11) are accurate approximations. We assume that P > 0. The sign of w can be posi- tive or negative. If w is positive, then the mutant allele's effect is to increase the intensity of policing. If w is negative, then the mutant alleles effect is to decrease the intensity of policing. Note that this formalism could also be interprets as describing the case of weak penetrance, in which only a small fraction of all workers that have the mutant genotype express the mutant phenotype. For considering the dynamics of a dominant police allele with weak phenotypic mutation, we introduce the quantity Pl tI-Ma Cdom tiro, 112\ ro 12 (n12) k ro re C m)] (12) - 2 If C.,„„ > 0, then increased intensity of policing is selected, and if Cam <0. then increased intensity of policing is not selected. This is just a different way of writing ( I ). We substitute (11) into (121 and collect powers of w. To first order in w. we get CdomWrn -2)P R R+ 2(2+n + nP)ri ÷O(W2) (13) 4n For considering the dynamics of a recessive police allele with weak phenotypic mutation, we introduce the quantity titan 2(2+ n + npo) Cm - (14) ro (2 + n)(2 + po) + plia„,(n - 2) If Cm( > 0, then increased intensity of policing is selected, and if Cmc <0, then increased intensity of policing is not selected. This is just a different way of writing (9). We substitute ( I I) into (14) and collect powers of w. To first order in w, we get - 2), R +2(2 + n + nP)K] Crec m W +O(1V2) (15) 4nRa 4.n+ n11) Notice that ( 13) and (15) are, up to a multiplicative factor, the same to first order in w. Using Eqs. (13) and (IS), the condition for policing to increase from a given level Z is /rat > —(n-2) R(Z) 19Z) 2(2 + n + nPan (16) Policing decreases from a given level Z if the opposite inequality holds. We have explicitly written the 2dependencies in Eq. (16) to emphasize that the quantities P.P. R. and /V are all functions of the intensity of policing. Z. The left-hand side of Eq. (16) can be understood as a ratio of marginal effects. To be specific, the left-hand side gives the ratio of the marginal change in efficiency over the marginal increase in the proportion of queen-derived males, if policing were to increase by a small amount For selection to favor increased policing, this ratio of marginals must exceed a quantity depending on the current values of 12 and P. Notice that the sign of the right-hand side is determined by n —2. So we get different behavior for different numbers of matings: • For n =2 (double mating), policing increases from Z if and only if K(Z) > 0. This means that evolution maximizes the value of R. regardless of the behavior of P. In other words, for double mating, evolution maximizes colony efficiency regardless of the effect on the number of queen-derived males. • For n=1 (single mating), the right-hand side of Eq. (16) is positive. So the condition for Z to increase is more stringent than in the n=2 case. Increases in policing may be disfavored even if they increase colony efficiency. • For n z 3 (triple mating or more than three matings). the right- hand side of Eq. (16) is negative. So the condition for Z to increase is less stringent than in the n=2 case. Any increase in policing that improves colony efficiency will be favored, and even increases in policing that reduce colony efficiency may be favored. Eqs. (13) and (15) also allow us to determine the location(s) of evolutionarily singular strategies (Geritz et al., 1998). Intuitively, a singular strategy is a particular intensity of policing, denoted by Zs. at which rare workers with slightly different policing behavior are, to first order in w, neither favored nor disfavored by natural selection. The parameter measuring intensity of policing, Z. can take values between 0 (corresponding to no policing) and 1 (cor- responding to full policing). There are several possibilities: There may not exist a singular strategy for intermediate intensity of policing; in this case, there is either no policing (V = 0) or full policing (r =1). If there exists a singular strategy for 0 < < I. then there are additional considerations: There may be convergent evolution toward intensity r or divergent evolution away from intensity In a small neighborhood for which Z x2*. further analysis is needed to determine if the singular strategy corre- sponding to r is an ESS. To determine the location(s) of evolutionarily singular strate- gies, we set the quantity in square brackets that multiplies w in (131 and I to zero. yielding RV') We) +fn-21— 0 (17) P(r) 2(2-F n+riP(r)) Eq. (17) gives the location(s) of singular strategies for both domi- nant and recessive mutations that affect policing. For a given singular strategy V. there is convergent evolution toward Z* if d E IY(Z) ?W.° 2 ( 2 -i-i iiP(Z)).1 - z• < ° There is divergent evolution away from 2° if the opposite inequality holds. It is helpful to consider some examples. If the functions f(Z) and R(Z) are known for a given species. then the behavior of worker policing with gradual evolution can be studied. It is pos- sible that policing is at maximal intensity, Z. = 1 (Fig. 14(a)), is nonexistent, o (Fig. I4(b)). is bistable around a critical value of intensity. 0 <2* < 1 (Fig. 14(c)), or exists at an intermediate value of intensity, 0 <V < 1 (Fig. 14(d)). Note that a singular strategy may or may not be an evolutio- narily stable strategy (ESS). (For example. it is possible that there is convergent evolution toward a particular singular strategy Zs which is not an ESS. In this case, once Z 2', evolutionary E F TA_R 1_02076889 EFTA02702885 J.W. Olefarz n at /Journal of Theowrkol Biology 399 (2016)103- fie II) a There is full policing. (1-41) • 1.20 1.15 .= 1.10 g 1.05 1 02 0.4 0.6 0.6 Intensity of policing. C • 120 • 1.15 1.10 8 1.05 - LT, 1 • E5 0.8 0.6 I I OA 3 0.2 00 Intensity of policing, 7 b 5 There is no poking. (Z=0) 1.20 1.15 1.10 1.05 1 S0.8 0.8 0.2 ° O 0.2 0.4 0.6 0.8 Intensity of poking, d There is imermediate policing. .0.7986...) 120 - 1.15 - I 1.10- 1.05 W 0.8 •E's: 0.6 n 04 8. 02 o 0.2 0.4 0.6 0.8 Intensity of policing. 2 Fig. 14. Several simple examples of functions F(2) and 11(2) are shown. For single mating the corresponding dynamics of policing Intensity with gradual evolution are also shown. We use the forms Pat= 1 - P +P7 and 2(21= I +Ca+ (1/2)C222. For eadi of the four panels. we set: (a) F" = 0.5. C. =0.2. Co =O. corresponding to Zs = I: (b) P - 0.8. CI -0.1, C2 - O. corresponding to r-tx (c) P -0.8, ei =0.12, C2 rP 0, corresponding to bistability around - 1/3: (d) P -OA. CI -02. C2 - - 018, corre- sponding to an intermediate level of policing around Z. w 0.7986.... branching may occur; Gen tz et al.. 1998) To determine if (17) is an ESS. we must look at second-order terms in (12) and (14). For a dominant police allele, we return to (12) with the sub- stitutions (11). We focus on a singular strategy given by (17). For a singular strategy, Cam, is zero to first order in w. To second order in w. we get Coon, w2 tn2 -41P.R2 + 2(n2 + 4n -4)Fiftl 16n2R2 + 8nPR'2 + 2012 + n2P+ 4001 +OW) 16n2R2 We may alternatively write (18) by substituting for ir using (17): [ Can,_ w2 (2 + n + nP)2I(n2 — 4),R + 20r2 + n2P+4fle] 16n2R(2+n+ni2)2 (n —4)(n2+Ii2P+4n-4)P2R1+000) (19) 16n2R(2+n+nP)2 For a recessive police allele, we return to (14) with the sub- stitutions (11). We focus on a singular strategy given by (17). For a singular strategy. Cm is zero to first order in w. To second order in w, we get Crec= w2 tn —2R2+ n+nP)P1—ur — 2)2PQR 16n2R(2+n+nP)l +16n2Ra+n+nP)2 2(2+n+ nP)2R* ] +0(Q) (20) Inspection of (18) and (20) allows us to determine if a singular strategy is an ESS. If the bracketed quantity multiplying 14,2 is negative, then mutations that change policing in either direction are disfavored. If the bracketed quantity multiplying w2 is positive, then mutations that change policing in either direction are favored. Thus, for a dominant allele that affects intensity of (18) policing, the singular strategy (17) represents a local ESS if (n2 - 4)1'R2+2(n2+ 4n-4) YR'R+8nPRI2 +2(n2 +n2P+4)/eR <0 (21) We may alternatively write (21) by substituting for /11 using (17): (2 +n+nP)21(n2 — 4), 12+2(r,2 +rr2P+4),I — (n2 —4)(n2+ n2P+4n —4)122R < 0 (22) Similarly, for a recessive allele that affects intensity of policing, the singular strategy (17) represents a local ESS if (n-2X2+n+nPfR—(n— 2)2112R +2(2 +n+nP)211 <0 (23) Here. R P, P, R. fe. and le are all functions of the intensity of policing, Z. The local ESS conditions (22) and (23) are quite opaque and do not allow for simple analysis. Notice that, although the locations of evolutionarily singular strategies are the same for dominant and recessive mutations that influence policing, the conditions for a singular strategy to be a local ESS are different. 9. Policing and inclusive fitness theory It has been claimed that policing is a test case of inclusive fit- ness theory (Abbot et .11.. 2011). But the first two papers to theo- retically establish the phenomenon (Woyciechowski and Lomnicki. 1987: Ratnicks. 1988) use standard population genetics; they do not mention the term "inclusive fitness", and they do not calculate inclusive fitness. Therefore, the claims that theoretical investiga- tions of worker policing emerge from inclusive fitness theory or that empirical studies of policing test predictions of inclusive fit- ness theory are incorrect. In light of known and mathematically proven limitations of inclusive fitness theory (Nowak et al.. 2010; Allen et al.. 2013), it is unlikely that inclusive fitness theory can be used to study general E F TA_R 1_02076890 EFTA02702886 114 1W. Olden et al /journal of Theoretkal &elegy 399 (2016) 103-116 questions of worker policing. Inclusive fitness theory assumes that each individual contributes a separate, well-defined portion of fitness to itself and to every other individual. It has been shown repeatedly (Cavalli-Sforza and Feldman. 1978; Uyenoyama and Feldman, 1982; Matessi and Karlin. 1984; Nowak ct al., 2010; van Veelen et al.. 2014). that this assumption does not hold for general evolutionary processes. Therefore, inclusive fitness is a limited concept that does not exist in most biological situations. Our work shows that the evolution of worker policing depends on the effectiveness of egg removal (pz) and the consequences of colony efficiency (r2). Each of these effects can be nonlinear (not the sum of contributions from separate individuals), with impor- tant consequences for the fate of a policing allele. Moreover, the invasion and stability conditions involve the product of p- and r- values, indicating a nontrivial interaction between these two effects which does not reduce to a simple sum of costs and ben- efits. We also found that there are separate conditions for invasion and stability. with neither implying the other. Inclusive fitness theory, which posits a single, linear condition for the success of a trait, is not equipped to deal with these considerations. Attempts to extend inclusive fitness theory to more general evolutionary processes (Queller. 1992; Frank. 1983; Gardner et al.. 2011) rely on the incorrect interpretation of linear regression coefficients (Allen et al.. 2013; see also Birch and Okasha. 2014). This misuse of statistical inference tools is unique to inclusive fitness theory. and differs from legitimate uses of linear regression in quantitative genetics and other areas of science. It was also recently discovered that even in situations where inclusive fitness does exist, it can give the wrong result as to the direction of nat- ural selection (Tarnita and Taylor, 2014). Relatedness-based arguments are often seen in conjunction with inclusive fitness but there is a crucial difference. Consider the following statement: if the queen is singly mated, then workers share more genetic material with sons of other workers than with sons of the queen. This statement is not wrong and could be useful in formulating evolutionary hypotheses. Such hypotheses can then be checked using exact mathematical methods. The problem arises when one attempts to formulate the quantity of inclusive fitness by partitioning fitness into contribu- tions from different individuals and reassigning these contribu- tions from recipient to actor. A worker does not make separate contributions to fitnesses of others. and therefore does not have Inclusive fitness". Arguments such as "the worker maximizes her inclusive fitness by not policing" are meaningless, since they are based on maximizing a nonexistent quantity. Moreover, even when evolution leads individuals to maximize some quantity, that quantity is not necessarily inclusive fitness (Okasha and Martens. 2015; Lehmann et al., 2015), It is true that genes (alleles) can be favored by natural selection if they enhance the reproduction of copies of themselves in other individuals. But that argument works out on the level of genes and can be fully analyzed using population genetics. Inclusive fitness only arises when the individual is chosen as the level of analysis. which is a problematic choice for many cases of complex family or population structure (Akcay and Van Clew. 2016). Bourke (20111 has proposed that inclusive fitness remains valid as a concept even when it is nonexistent as a quantity. But why is such an uninstantiable concept useful? The mathematical theory of evolution is clear and powerful. Exact calculations of evolutionary dynamics (Antal et al.. 2009; Allen and Nowak. 2014; Fu et al.. 2014; Haven and Doebelt, 2004; Szabo and Kith. 2007; Antal and Scheming. 2006; Traulsen et al., 2008; van Veelen et al.. 2014; Simon ct al.. 2013) demonstrate that inclusive fitness is not needed for understanding any phenomenon in evolutionary biology. This realization is good news for all whose primary goal is to understand evolution rather than to insist on a particular method of analysis. By releasing ourselves from the confines of a mathematically limited theory, we expand the possibilities of scientific discovery. it Discussion We have derived analytical conditions for the invasion and stability of policing in situations where queens mate once or several times and where colony efficiency can be affected by policing. In the special case where policing has no effect on colony efficiency, our results confirm the traditional view that policing does not evolve for single mating, is neutral for double mating, and does evolve for triple mating or more than three matings. If colony efficiency depends linearly or monotonically on the fraction of workers that are policing, then our results support the view that multiple mating is favorable to evolution of policing (Ratnieks. 1988). Our results also show that non-monotonic relations in colony dynamics and small changes in colony efficiency necessi- tate a more careful analysis. We find that policing can evolve in species with singly mated queens if it causes minute increases in colony efficiency. We find that policing does not evolve in species with multiply mated queens if it causes minute decreases in colony efficiency. For non- monotonic efficiency functions, it is possible that single mating allows evolution of policing while multiple mating opposes evo- lution of policing. Our analysis is the first to give precise conditions for both the invasion and stability of policing for both dominant and recessive mutations that effect policing. We study the evolutionary invasion and evolutionary stability of policing both analytically and numerically. For any number of matings, there are four possible outcomes (see Fig. 5): (i) policing can invade and is stable; (ii) policing can invade but is unstable, leading to coexistence; (iii) policing cannot invade but is stable, leading to bistability; (iv) policing cannot invade and is unstable. We give precise conditions for all outcomes for both dominant and recessive police alleles. All outcomes can be achieved with arbitrarily small changes in colony efficiency. Our calculations are not based on any assumption about the strength of phenotypic mutation induced by an allele. The condi- tions I . . 91. and (10) also describe the dynamics of mutations that have an arbitrarily small phenotypic effect on colony dynamics. This facilitates investigation of the evolution of complex social behaviors that result from gradual accumulation of many mutations (Kapheim et al.. 2016). We derive a simple relation. Eq. (17), for the location(s) of evolutionarily singular strategies. We also derive precise conditions for a singular strategy to be an ESS. These results are applicable for understanding both the case of weak phenotypic effect and the case of weak penetrance. Our analysis does not use inclusive fitness theory. Given the known limitations of inclusive fitness (Nowak et al.. 2010; Allen et al.. 2013). it is unlikely that inclusive fitness theory could provide a general framework for analyzing the evolution of worker policing. In summary, the main conclusions of our paper are: (i) The prevalent relatedness-based argument that policing evolves under multiple mating but not under single mating is not robust with respect to arbitrarily small variations in colony efficiency; (ii) for non-monotonic efficiency functions, it is possible that policing evolves for single mating, but not for double or triple mating; (iii) careful measurements of colony efficiency and the fraction of queen-derived males are needed to understand how natural selection acts on policing; (iv) contrary to what has been claimed (Abbot et al.. 2011), the phenomenon of worker policing is no empirical confirmation of inclusive fitness theory: the first two mathematical papers on worker policing (Woyacchowski and lomnicki. 1987; Ratnieks. 1988) do not use inclusive fitness theory. EFTA_R1_02076891 EFTA02702887 J.W. °Wars et ol. / Journal of Theoretkal Biology 399 (20:6)103-n6 115 The present paper, which also does not use inclusive fitness the- ory. is the first detailed analysis of policing for any number of coatings and taking into account effects on colony efficiency. Aclmowledgments We are grateful to the referees and editor for helpful comments that have significantly benefited this manuscript. This publication was made possible through the support of a grant from the John Templeton Foundation. The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the John Templeton Foundation. Appendix A. Supplementary data Supplementary data associated with this paper can be found in the online version at http://dx.doi.org/10.1015/Kitbi.2016.03.001. References Abbot. C, er al.. 2011. Inclusive fitness theory and eusociality. Nature 471. E1-F9. Alkay. E. Van Cleve. J.. 2016. There is no fitness but fitness. and the lineage is its bearer. Philos Trans. R. Soc. 13371. 20150085. Allen. Et, Nowak. MA.. Wilson, E,0.. 2013. Limitations of inclusive fitness. Proc. Nail, Mad. 50. U.SA 110.20135-20139. Allen. B.. Nowak. MA„ 2014. Games on graphs. EMS Sun'. Math, Sci. I (I ). 115-151. natal. I.. Scheming. 2006. Fixation of strategies for an evolutionary game in finite populations. Bull, Math. Biol. 68, 1923-1944. Antal. T.. ()Musks It,. Wakeley. J.. Taylor. 1W.. Nowak. MA.. 2009. Evolution of cooperation by phenotypic similarity, Proc. Natl. Mad. Sci. USA. 106 (21), 8597 8600. Beekman. M.. Oldroyd, B.R. 2005. Honeybee workers use cues mho than egg via• Wiry (or policing. Biol. Lets I. 129-132. Birch. J.. Okasha S., 2014, Kin selection and its critics. koScience 65. 22-32. Bonckaert. W.. Vuefinekt. K. Billen, J., Hammond. RL. Kellen. L. Wenseleers. T.. 2008. Worker policing in the German wasp Vespula germaniro. Behay. Ecol. 19. 272-278. gonekarts W. van Zweden. JS„ &Entine. P.. Billets J. Wenseleers. L. 2011. Colony stage and not facultative policing explains pattern of worker reproduction in the Saxon wasp. Mol. Ecol. 20. 3455 3468. Boomuna. IL 2007. Kin selection versus sexual selection: why the ends do not Meet (WE Biol. 17. R673-K6133. BoomIma, J,f.. 2009. lifetime monogamy and the evolution of euuniality. Trans. R. Soc. B 364. 3191 3207. Bourke. A.F.G.. 2011. The validity and value of inclusive fitness theory. Proc. R. Soc. B 278. 3313-3320. Uvalli-Sforza. LI.. Feldman. MW.. 1978.1)aiwinian selection and lheor. Popul. Biol. 14. 268-280. Cole. Rt. 1986. the social behavior 01 Scow :home ollardynii wvenoptera. For micidae): rime budgets and the evolution of worker reproduction. Behay. Ecol. Sociothol, IS. 165-173. C.K. West, S.A., Davis. KF,., Griffin. AS., 20ln Promiscuity and the evolutionary transition to complex societies, Nature 466. 969-972. Dieckmann, U.. law, R.. 1996. The dynamical theory of coevolution: a derivation from stochastic ecological processes. J. Math. Biol. 34. 579-612. Endkr. A.. t.iebig. J.. Schmitt. T. Patter. It. Jones. G.K. Schroer, Holldobler. B.. 2004. Surface hydrocarbons of queen eggs regulate woiket teproduction in a social insect. Proc Natl. Mad. Sc,. U.S.A. 101. 2945-2950. Poster. KR.. Ratnieks. ELW.. 2000. Facultative worker policing in a wasp. Nature 407.6312-693. Foster. KR_ Raimeks. F.LW.. 2001a. Convergent evolution of worker policing by egg eating in the honeybee and common wasp. Proc. R. Soc. Land. 8268, 169-174. Foster. KR.. Ratmeks. F.LW.. 2001b. The effect of sex-allocation biasing on the evolution of worker policing in hymenopteran societies. Ant Nat. 158.615-624. Foster. K.R.. Rainglis r.t.w.. 2001c. Paternity. tepreduCtion and conflict in vespine wasps: a model system for testing kin selection predictions. &bar. Ecol. Sotiobiol. 50. 1-8. Foster, 1CR.. Gulliver. J.. Ratnieks. F.LW„ 2002. Worker policing in the European hornet Wspo Erato°. Insectes Soc. 49.41-44. FroMes K.R.. Wenseleers. T. Ratineks. F.LW. 2006. Kin selection is the key to altruism. trends Ecol. Ewa 21. 57-60. Frank. S,A.. 1983. Foundations of Social Evolution. Princeton University Press. Princeton. NJ. It. E. Wicker. S.D.. Nowak. MA. 2014. The risk-return trade-olf between solitary and eusonal trproduction. Fxol. Lett 19 (Ile 74-84. Gadagkar. It. 2001. 'Ilk Social Biology of Ropalidia hlargiluu: low.u4 Under- standing the Evolution of EuSOCiality. Harvard University Press. Cambridge. MA. Cadagkar. R, 2004. Why do honey bee workers destroy each other's eggs? J. Mout 29(3). 213-217. Cardneu. A.. West, SA. Wild. G.. 2011. The generical theory of kin selection. J. Evol. Riot 24. 1020-1043. C,eritz. SAIL. Kisdi. E., Meszena. G.. Metz. JAL 1998. Evolutionarily singular stra- tegies and the adaptive growth and branching of the evolutionary tree. Evol. Earl. 12. 35-57. Hamilton. W.D.. 1972. Altruism and 'elated phenomena, mainly in social Insects. Annu. Rev. Ecol. Syst. 31. 193 232. liammond, RI_ Keller. I.. not Conflict over male parentage in social insects. MOS Biol. 2 (e248). 1472 1482. Han:Tuna, A., Wantu, J.. 'Torres. JA„ Heiner. J., 2003. Worker policing without genetic conflicts in a clonal ant. Proc. Nail. Acad. So. USA IOU 12836-12840. Hawn. C.. Doebeli, M.. 2004. Spatial structure often inhibits the evolution of cooperation in the snowdrift game. Nature 428. 643-646. I ielantria. II.. TolfiSki. A.. Wenseleers. 1', %Hoicks. F.LW.. 2006. Worker policing in the common wasp liespncla sultans is not aimed al Improving colony hygiene. lusectes Soc. 53. 399-402. titian:era. H.. Sundstrons L. 2007. Worker reproduction in formica ants. Am. Nat. 170. E14-F25. Ifolbaues J.. Sigmund. le. 1990. Adaptive dynamics and evolutionary stability. Appl. Math. Lett. 3. 75 79. Hughes. W,O.H.. Oldroyd. lilt. Beekman. M.. Ratnieks. FLW.. 2008. Ancestral monogamy shows kin selection is key to the evolution of eusociality. Science 320. 1213-1216. Kapheim. KM.. Pan, /I, ti. C.. SalzIwig. S,1-, Ruin 1).. et al„ 2015. Cenomic signal ores of evolutionary riansitions from solitary to group living. Science 348. 1139-1143. Khila, A.. Abouheif. E.. 2008. Reproductive constraint is a developmental mechan- ism that maintains social harmony in advanced ant societies. Proc. Natl. Acad. Sci. U.S.A. 105. 17889-17889. Lehmann. I., Alger. I.. Weihull, J.. 2015. Does evolution lead to maximizing beha- vior? Evolution 60 1858 1873. Matessi, C. Karlin. S.. 1984.0n the evolution of altruism by kin selection. Proc. Natl. Acad. So. U.SA. 81. 1754 1758, Metz. J.A.J. Cecil,. SA.11. Meszena. C. Jacobs. F.JA. van Ikerwaarden. IS.. 1996. Adaptive dynamics, a geometrical slimly of the consequences of nearly faithful repregluction. In: van Sums S.J.. Verduyn Luna S.M. (Eds.). Stochastic and Spatial Structures of Dynamical Systems. KNAW Verhandelingen. Afel.. Amsterdam. pp. 183-231. Nancy. N.L.. Peso, M.. Even. N.. Barron. A.B.. Robinson. G.E.. 2013. Altivistic behavior by egg-laying worker honeybees. Curt. Biol. 23. 1574-1578. Nonacs, It. 2006. Nepotism and brood reliability in the suppression of worker reproduction in the emocul hymenoptera. Biol. lett. 2, 577-574. Nowak. MA. Sigmund, K.. 1990. The evolution of stochastic strategies in the pris- oner's dilemma. Acta Appl. Math. 20. 247 265. Nowak, MA., tonna. Cl., Wilson. CO.. 2010. The evolution of eusociality. Nature 466,1057-1062. Okasha, S.. Martens. L. 2016. Hamilton's rule, inclusive fitness maximization. and the goal of individual behaviour in symmetric two-prayer games. J. Evol. Biol. 29. 473-482. Dicker, J.W„ Allen. B., Vellet, C., Nowak, MA, 2015. The evolution of non reproductive workers in insect colonies with haplocliplold genetics elite 4. e08918. Peters. JAL. Quelkr. D.C.. Imperatriz-Fonseca. V.L. Roulade. D.W_ Strassmann. 1999. Mate number, kin selection and social conflicts in stingless bees and honeybees, Proc. R. Soc. loud. B 266. 379 384. Pint, C.W.W., Neumann, P. Hepburn. R.. hloritz. R.I.A.. Tautr. J.. 1999. Fgg viability and worker policing in honey bees. Proc. Natl. Acad. Sol. USA 101, 8649 8651. Queller. D.C.. 1992. A general model for kin selection. Evolution 46. 376-380. Queller, D.C., Strassmann, J.E.. 1998, Kin selection and .4(ICUI insects. Rinscience 48. 165-175. Ratnieks. F.I.W., 1988. Reproductive harmony via mutual policing by workers in cusocial hymenoptera. Ant Nat. 132. 217-236. Ratnieks. FLW..1Asscher. P.K. 1989. Worker policing in the honeybee. Nature 342. 796-797. Ratrucks. ELW., Foster. Kit, Wenseleers, T. 2006. Conflict resolution in insect societies. Annu. Rev. Entomol. SI, 581-608, Seeley. 1.D.. 1985.11oneybee Ecology: A Study of Adaptation in Social Life. Princeton University Press. Princeton. NJ. Simon, B. l'sr:Int. IA.. Doeben M. 7013 general theory of group stir • • • • , h7, 1561- I572. Start C.K., 1984. In. Smith. R.L (Ed.). Sperm Competition. Kinship• and Sociality in the Acoleate I Ivmenoplera. Academic Press. Orlando, Florida. pp. 427-464. Szabo. C., .' Evolutionary gum. tin glimils Illy. Rep. 446. 97 216. Tamita, C.E. Taylor. P.D. 2014. Measures of relative fitness of social behaviors in finite structured population models. Am. Na. 184(4) in press. Iraulsen. A., Shoreils N. Nowak, M A. 2008. Analytical results for individual and group selection of any intensity. MI. Math. Biol. 70, 1410-1424. Dyenoyania, M.K., Feldman, M.. 1982. Population genetic theory of kin selection. II, The multiplicative model. Am. Nat. 120, 614 627. van Verlen. M., WO. S.. Simon. 8.. 2014. A simple model of group selection that unnol be analyzed with inclusive fitness. J. lheor. Biol. 360. 279-289. EFTA_R1_02076892 EFTA02702888 116 J.W. °kiwi et al /Journal of Theoretical Biology 399(2016)103-116 Winans, H.H.W.. de Anus, Alves, Lt. Impeistinz-lonseca. V.L. Jose. M.. 2002. Worker bees and the (ate of their eggs. Proc. Exp. Appl. Eniomol. NEV Amsterdam 13. 97-102. Wenseleers. T.. Helantera, H_ Han. A. Ratnieks. F.LW..200.4a. Winker reproduction and policing in insert societies: an FS5 analysis. J. (vol. Riot. 17. 1035-1047. Wenseleers. T. Han. A6. Ratnieks. F.LW.. 2004b. When sesistaixe is useless: policing and the evolution of reproductive acquiescence in insect societies. Am. Nat. 164, E154 E167. Wenseleers. T.. &Wrack. N.S. given. K.. tofilski. A, Nascimenio. ES., Hari. A.G.. Burke. TA. Archer. Ratnieks. 2005. A test of worker policing theory in an advanced eusocul wasp, Ye:sputa ndo. Evolution 59. 1306-1314. Wenseleers. T.. Ratnieks. F.LW., 2006a. Enforced altruism in insect societies. Nature 444. 50. Wenseleers, r, Itatowks. [LW.. 20066. Comparative analysis of worker reproduc- tion and policing in eusocul hyownopteia supports relatedness theory. Am. Nat. 168. E163-E179. Wenseleers, T.. Itelantera. H.. Alves. U.A., Otienez-Cusman. E.. ',ample. P.. 2013. Towards greater realism in inclusive fitness models: the case of worker reproduction in insect societies. lhol. let'. 9, 20130334. Wilson. Ell.. 1971. the Insect Societies. Harvard trowel soy Press. Cambridge, MA. tArowierhowski. M.. lommtki. A.. 1987. Multiple mating or queens and the sterility of workers among eusocial hymenoptera. J. 'fluor. Riot. 128. 317 327. 2anette. IRS. Miller. 5.0.1. Stria, C.M.A. Almond. E.J.. Huggins. T.1.. Jordan. W.C.. Boinke.A.F.C.. 2012. Reproductive conflict in bumblebees and the evolution of worker policing. Evolution 66, 3765-3777. EFTA_R1_02076893 EFTA02702889