DS9 Document EFTA00803978

1 a 1 Indirect reciprocity with private, noisy, and incomplete information Christian Hilbe11, Laura Schmid', Josef Tkadleca, Krishnendu Chatterjeee, and Martin A. Nowak" 'Institute of Science and Technology Austria, 3400 Klostemeuburg, Austria; °Program for Evolutionary Dynamics, Harvard University, Cambridge, MA 02138; 'Department of Organismic and Evolutionary Biology, Harvard University, Cambridge, MA 02138; and °Department of Mathematics, Harvard University, Cambridge, MA 02138 Edited by Brian Skyrms, University of California, Irvine, CA, and approved October 16, 2018 (received for review June 19, 2018) Indirect reciprocity is a mechanism for cooperation based on shared moral systems and individual reputations. It assumes that members of a community routinely observe and assess each other and that they use this information to decide who is good or bad, and who deserves cooperation. When information is trans- mitted publicly, such that all community members agree on each others reputation, previous research has highlighted eight cru- cial moral systems. These "leading-eight" strategies can maintain cooperation and resist invasion by defectors. However, in real populations individuals often hold their own private views of oth- ers. Once two individuals disagree about their opinion of some third party, they may also see its subsequent actions in a different light Their opinions may further diverge over time. Herein, we explore indirect reciprocity when information transmission is pri- vate and noisy. We find that in the presence of perception errors, most leading-eight strategies cease to be stable. Even if a leading- eight strategy evolves, cooperation rates may drop considerably when errors are common. Our research highlights the role of reli- able information and synchronized reputations to maintain stable moral systems. cooperation I indirect reciprocity I social norms I evolutionary game theory H umans treat their reputations as a form of social capital (1-3). They strategically invest into their good reputa- tion when their benevolent actions are widely observed (4-6), which in turn makes them more likely to receive benefits in subsequent interactions (7-12). Reputations undergo constant changes in time. They are affected by rumors and gossip (13), which themselves can spread in a population and develop a life of their own. Evolutionary game theory explores how good reputations are acquired and how they affect subsequent behav- iors, using the framework of indirect reciprocity (14-17). This framework assumes that members of a population routinely observe and assess each other's social interactions. Whether a given action is perceived as good depends on the action itself, the context, and the social norm used by the population. Behaviors that yield a good reputation in one society may be condemned in others. A crucial question thus becomes: Which social norms are most conducive to maintain cooperation in a population? Different social norms can be ordered according to their com- plexity (18) and according to the information that is required to assess a given action (19, 20). According to "first-order norms," the interpretation of an action depends only on the action itself. When a donor interacts with a recipient in a social dilemma, the donor's reputation improves if she cooperates, whereas her reputation drops if she defects (21-26). Accord- ing to "second-order norms," the interpretation of an action additionally depends on the reputation of the recipient. The recipient's reputation provides the context of the interaction. It allows observers to distinguish between justified and unjustified defections (27-29). For example, the standing strategy consid- ers it wrongful only to defect against well-reputed recipients; donors who defect against bad recipients do not suffer from an impaired reputation (30). According to "third-order norms,- observers need to additionally take the donor's reputation into account. In this way, assessment rules of higher order are increas- ingly able to give a more nuanced interpretation of a donor's action, but they also require observers to store and process more information. When subjects are restricted to binary norms, such that repu- tations are either "good" or "bad,- an exhaustive search shows there are eight third-order norms that maintain cooperation (20, 31). These "leading-eight strategies" are summarized in Table 1, and we refer to them as LI-IS. None of them is exclu- sively based on first-order information, whereas two of them (called "simple standing" and "stem judging," refs. 32 and 33) require second-order information only. There are several uni- versal characteristics that all leading-eight strategies share. For example, against a recipient with a good reputation, a donor who cooperates should always obtain a good reputation, whereas a donor who defects should gain a bad reputation. The norms dif- fer, however, in how they assess actions toward bad recipients. Whereas some norms allow good donors to preserve their good standing when they cooperate with a bad recipient, other norms disincentivize such behaviors. Ohtsuki and Iwasa (20, 31) have shown that if all members of a population adopt a leading-eight strategy, stable cooperation can emerge. Their model, however, assumes that the players' images are synchronized; two population members would always agree on the current reputation of some third population member. The assumption of publicly available and synchronized information Significance Indirect reciprocity explores how humans act when their rep- utation is at stake, and which social norms they use to assess the actions of others. A crucial question in indirect reciprocity is which social norms can maintain stable cooper- ation in a society. Past research has highlighted eight such norms, called "leading-eight" strategies. This past research, however, is based on the assumption that all relevant infor- mation about other population members is publicly available and that everyone agrees on who is good or bad. Instead, here we explore the reputation dynamics when information is private and noisy. We show that under these conditions, most leading-eight strategies fail to evolve. Those leading- eight strategies that do evolve are unable to sustain full cooperation. Author contributions: C.M., LS, J.T., K.C., and M.A.N. designed research performed esearch analyzed data. and wrote the paper. The authors declare no conflict of interest. This article is a PNAS Direct Submission. Published under the PNAS license. 'To whom correspondence should be addressed. Email: christian.hilbellistac.at. ThIsarlide contains supporting informationonlineat vwnvpnasorgAookuprsuPPIldoi:I6 1073/peas.111105651I9-itiCSupplementat viww.pnas.orgicgildoV10.10731pnas.1810S6S1 S PNAS Latest Articles I t et 6 EFTA00803978 1 Table 1. The leading-eight strategies of indirect reciprocity Assessment rule C Cs al C L a 2 a-2 5 co B UN (nu) LI L2 L3 L4 L7 L8 Good cooperates with Good Good cooperates with Bad Bad cooperates with Good Bad cooperates with Bad Good defects against Good Good defects against Bad Bad defects against Good Bad defects against Bad Action rule 9 9 9 9 9 9 9 9 9 b g g b b g b g g g g g g g g g g g b g b b b b b b b b b b 9 9 9 9 9 9 9 9 b b b b b b b b b o n g g b b LI L2 L3 L4 LS 16 L7 L8 Good meets Good Good meets Bad Bad meets Good Bad meets Bad C D C C C D C C C C C D C D C D C D C C C C D C D C There are eight strategies, called the Pleading eight* that have been shown to maintain cooperation under public assessment (20, 31). Each such strategy consists of an assessment rule and of an action rule. The assessment rule determines whether a donor is deemed good (9) or bad (b). This assess- ment depends on the context of the interaction (on the reputations of the donor and the recipient) and on the donor's action (C or D). The action rule determines whether to cooperate with a given recipient when in the role of the donor. A donor's action may depend on her own reputation, as well as on the reputation of the recipient. All of the leading-eight strategies agree that cooperation against a good player should be deemed as good, whereas defection against a good player should be deemed bad. They disagree in how they evaluate actions toward bad recipients. greatly facilitates a rigorous analysis of the reputation dynam- ics. Yet in most real populations, different individuals may have access to different kinds of information, and thus they might disagree on how they assess others. Their opinions may well be correlated, but they will not be correlated perfectly. Once individuals disagree in their initial evaluation of some person, their views may further diverge over time. How such initial dis- agreements spread may itself depend on the social norm used by the population. While some norms can maintain coopera- tion even in the presence of rare disagreements, other norms are more susceptible to deviations from the public information assumption (34-37). Here, we explore systematically how the leading-eight strategies fare when information is private, noisy, and incomplete. We show that under these conditions, most leading-eight strategies cease to be stable. Even if a leading- eight strategy evolves, the resulting cooperation rate may be drastically reduced. Results Model Setup. We consider a well-mixed population of size N. The members of this population are engaged in a series of cooperative interactions. In each round, two individuals are ran- domly drawn, a donor and a recipient. The donor can then decide whether to transfer a benefit b to the recipient at own cost c, with 0C c< b. We refer to the donor's two possible actions as cooperation (transferring the benefit) and defection (not doing anything). Whereas the donor and the recipient always learn the donor's decision, each other population mem- ber independently learns the donor's decision with probability q> 0. Observations may be subject to noise: We assume that all players who learn the donor's action may misperceive it with probability c > 0, independently of the other players. In that case, a player misinterprets the donor's cooperation as defection or, conversely, the donor's defection as cooperation. After observ- ing an interaction, population members independently update their image of the donor according to the information they have (Fig. 1). To do so, we assume that each individual is equipped with a strategy that consists of an assessment rule and an action rule. The player's assessment rule governs how players update the rep- utation they assign to the donor. Here we consider third-order assessment rules. That is, when updating the donor's reputa- tion, a player takes the donor's action into account, as well as the donor's and the recipient's previous reputation. Importantly, when two observers differ in their initial assessment of a given donor, they may also disagree on the donor's updated reputa- tion, even if both apply the same assessment rule and observe the same interaction (Fig. IC). The second component of a player's strategy, the action rule, determines which action to take when chosen to be the donor. This action may depend on the player's own reputation, as well as on the reputation of the recipient. A player's payoff for this indirect reciprocity game is defined as the expected benefit obtained as a recipient, reduced by the expected costs paid when acting as a donor, averaged over many rounds (see Materials and Methods for details). Analysis of the Reputation Dynamics. We first explore how differ- ent social norms affect the dynamics of reputations, keeping the strategies of all players fixed. To this end, we use the concept of image matrices (34-36). These matrices record, at any point in time, which reputations players assign to each other. In Fig. 2 A-H, we show a snapshot of these image matrices for eight dif- ferent scenarios. In all scenarios, the population consists in equal proportions of a leading-eight strategy, of unconditional cooper- ators who regard everyone as good (ALLC) and of unconditional defectors who regard everyone as bad (ALLD). Depending on the leading-eight strategy considered, the reputation dynamics in these scenarios can differ considerably. First, for four of the eight scenarios, a substantial propor- tion of leading-eight players assigns a good reputation to ALLD players. The average proportion of ALLD players considered A kale r•pgilitlaill I a) , 29 4 I 3, I Play- I Payer 2 Pivot 3 24 3g 191 B Poem mega C Uptlean1 notitelicee a; ;:- 30 30 Pest 2 li M coopleatOS aw I Pleiyy Mayer 2 Km% 3 Fig.'. Under indirect reciprocity, individual actions are continually assessed by all population members. (A) We consider a population of different players. All players hold a private repository where they store which of their coplayers they deem as either good (g) or bad (b). Different play- en may hold different views on the same coplayer. In this example, player 2 is considered to be good from the perspective of the first two play- en, but he is considered to be bad by player 3. (8) In the action stage, two players are randomly chosen, a donor (here, player 1) and a recipient (here, player 2). The donor can then decide whether or not to cooper- ate with the recipient. The donor's decision may depend on the stored reputations in her own private repository. (C) After the action stage, all players who observe the interaction update the donors reputation. The newly assigned reputation may differ across the population even if all players apply the same social norm. This can occur when individuals already disagreed on their initial assessments of the involved players, (if) when some subjects do not observe the interaction and hence do not update the donors reputation accordingly, or (iii) when there are percep- tion errors. 2 of 6 I vavw.pnas.orgrcgi/dolff0.I073/0nas.18105651t5 Hilbe et al. EFTA00803979 as good by L.3, L4, 15, and 1.6 is given by 31%, 31%, 42%, and 50%, respectively (SI Appendix, Fig. SI). In terms of these four leading-eight strategies, a bad player who defects against another bad player deserves a good reputation (Table 1). In particular, ALLD players can easily gain a good reputation whenever they encounter another ALLD player. Moreover, the higher the proportion of ALLD players in a population, the more readily they obtain a good reputation. This finding suggests that while 13-Lb might be stable when these strate- gies are common in the population (20, 38), they have prob- lems in restraining the payoff of ALLD when defectors are predominant. Second, leading-eight players may sometimes collectively judge a player of their own kind as bad. In Fig. 2, such cases are represented by white vertical lines in the upper left square of an image matrix. In SI Appendix, Fig. S2 we show that such apparent misjudgments are typically introduced by perception errors. They occur, for example, when a leading-eight donor defects against an ALLC recipient, who is mistakenly considered as bad by the donor. Other leading-eight players who witness this interaction will then collectively assign a bad reputation to the donor—in their eyes, a good recipient has not obtained the help he deserves. This example highlights that under private infor- mation, an isolated disagreement about the reputation of some population member can have considerable consequences on the further reputation dynamics. To gain a better understanding of such cases, we analytically explored the consequences of a single disagreement in a homo- geneous population of leading-eight players (see SI Appendix for all details). There we assume that initially, all players con- sider each other as good, with the exception of one player who considers a random coplayer as bad. Assuming that no further errors occur, we study how likely the population recovers from this single disagreement (i.e., how likely the population reverts to a state where everyone is considered good) and how long it takes until recovery. While some leading-eight strategies are guaranteed to recover from single disagreements, we find that other strategies may reach an absorbing state where players mutually assign a bad reputation to each other. Moreover, even if recovery occurs, for some strategies it may take a consider- able time (SI Appendix, Fig. S3). Two strategies fare particularly badly: 1.6 and 1.8 have the lowest probability to recover from a A LI OLLC NW LI ALLC ALLD E LS ALLC ALLD LS NW LO B Lx AMC ALM F L6 ALLC ALLO L3 M1C MID La ALLC ALLD L6 ALLC ALLD single disagreement, and they have the longest recovery time. This finding is also reflected in Fig. 2, which shows that these two strategies are unable to maintain cooperation. L6 eventually assigns random reputations to all coplayers, whereas 1.8 assigns a bad reputation to everyone (SI Appendix, Fig. S4). While 1.6 ("stern") has been found to be particularly successful under pub- lic information (18, 32, 33), our results confirm that this strategy is too strict and unforgiving when information is private and noisy (34-36). Evolutionary Dynamics. Next we explore how likely a leading-eight strategy would evolve when population members can change their strategies over time. We first consider a minimalistic sce- nario, where players can choose among three strategies only, a leading-eight strategy L„ ALLC, and ALLD. To model how play- ers adopt new strategies, we consider simple imitation dynamics (39-42). In each time step of the evolutionary process, one player is picked at random. With probability p (the mutation rate), this player then adopts some random strategy, corresponding to the case of undirected learning. With the remaining prob- ability 1— p, the player randomly chooses a role model from the population. The higher the payoff of the role model, the more likely it is that the focal player adopts the role model's strategy (Materials and Methods). Overall, the two modes of updating, mutation and imitation, give rise to an ergodic process on the space of all population compositions. In the following, we present results for the case when mutations are relatively rare (43, 44). First, we calculated for a fixed benefit-to-cost ratio of 61 e= S how often each strategy is played over the course of evolu- tion, for each of the eight possible scenarios (Fig. 3). In four cases, the leading-eight strategy is played in less than 1% of the time. These cases correspond to the four leading-eight strate- gies L3—L6 that frequently assign a good reputation to ALLD players. For these leading-eight strategies, once everyone in a population has learned to be a defector, players have difficul- ties in reestablishing a cooperative regime (in Fig. 3 C—F, once ALLD is reached, every other strategy has a fixation probabil- ity smaller than 0.001). In contrast, the strategy 1.8 is played in substantial proportions. But in the presence of noise, players with this strategy always defect, because they deem everyone as bad (Fig. 2). C AILC MID D L4 MSC NW ALLC H LE ARLO ALLD La ALLC OLID Fig. 2. (A-H) When individuals base their decisions on noisy private information, their assessments may diverge. Models of private information need to keep track of which player assigns which reputation to which coplayer at any given time. These pairwise assessments are represented by image matrices. Here, we represent these image matrices graphically, assuming that the population consist of equal parts of a leading-eight strategy, of unconditional cooperators (ALLC) and unconditional defectors (ALLD). A colored dot means that the corresponding row player assigns a good reputation to the column player. Without loss of generality, we assume that ALLC players assign a good reputation to everyone, whereas ALLD players deem everyone as bad. The assessment of the leading-eight players depend on the coplayer's strategy and on the frequency of perception errors. We observe that two of the leading- eight strategies are particularly prone to errors: L6 ("stem judging") eventually assigns a random reputation to any coplayer, while 18 ("judging') eventually considers everyone as bad. Only the other six strategies separate between conditionally cooperative strategies and unconditional defectors. Each box shows the image matrix after 2 .104 simulated interactions in a population of size N = 3.30 = 90. Perception errors occur at rate e = 0.05, and interactions are observed with high probability, q = 0.9. Hilbe et al. PROS Latest Articles I 3 el 6 EFTA00803980 '44* A AlLD ALLC E LI 400. . 4.3%': 1 <0.001. .0.031 .0001 aide 1 • 99.5% <0001. /IUD ALLC Consistent Stancing 001? c0501 0000 0.130 COM 0.3% MID ALLC F Stern Judong ts • ..001 0A0,/ 1000% `000‘. 00% N. OW ALLD MAC SMDIO Staicirg C 13 D 4691 0090 eStel T 0.,21 <0.00, 0012\ coil 0409 h _-!. 4 / \ •401. 0.1% 'MI, 0,0% ALLD NLC MID MSC Sia)Ing 1% COI.. 55% Ot69 ALLD 3010 Jul;ing L8 0152 0020; WON 10.020, 0,169 \ V '50.0% `00/... 0,0% 0%9_ ALL0 ALIO Fig. 3. Most of the leading-eight strategies are disfavored in the presence of perception errors. We simulated the evolutionary dynamics when each of the leading-eight strategies competes with ALLC and ALLO. These simulations assume that, over time, players tend to imitate coplayers with more profitable strategies and that they occasionally explore random strategies (Materials and Methods). The numbers within the circles represent the abundance of the respective strategy in the selection-mutation equilibrium. The numbers close to the arrows represent the fixation probability of a single mutant into the given resident strategy. We use solid lines for the arrows to depict a fixation probability that exceeds the neutral probability 1/N, and we use dotted lines if the fixation probability is smaller than 1/N. In four cases, we find that ALL° is predominant (C4). In one case (H), the leading-eight strategy coexists with ALM. but without any cooperation. In the remaining cases (A, 8, and G), we find that LI and L7 are played with moderate frequencies, but only populations that have access to 12 (*consistent standing') settle at the leading-eight strategy. Parameters: Population size N= 50, benefit b = S, cost c = 1, strength of selection s =1, error rate e = 0.05. observation probability q = 0.9, in the limit of rare mutations p 0. There are only three scenarios in Fig. 3 that allow for positive cooperation rates. The corresponding leading-eight strategies are LI, 12 ("consistent standing"), and L7 ("staying,- ref. 45). For LI and L7, the evolutionary dynamics take the form of a rock-scissors-paper cycle (46-50). The leading-eight strategy can be invaded by ALLC, which gives rise to ALLD, which in turn leads back to the leading-eight strategy. Because ALLD is most robust in this cycle, the leading-eight strategies are played in less than one-third of the time (Fig. 3A and C). Only consistent standing, I.2, is able to compete with ALLC and ALLD in a direct comparison (Fig. 38). Under consistent standing, there is a unique action in each possible situation that allows a donor to obtain a good standing. For example, when a good donor meets a bad recipient, the donor keepsv her good standing by defecting, but loses it by cooperating. Compared with stem judging, which has a similar property (18), consis- tent standing incentivizes cooperation more strongly. When two bad players interact, the correct decision according to consistent standing is to cooperate, whereas a stern player would defect (Table I). Nevertheless, we find that even when consistent standing is common, the average cooperation rate in the population rarely exceeds 65%. To show this, we repeated the previous evolution- ary simulations for the eight scenarios while varying the benefit- to-cost ratio, the error rate, and the observation probability (Fig. 4). These simulations confirm that five of the leading-eight strategies cannot maintain any cooperation when competing with ALLC and ALLD. Only for LI, L2, and L7 are average coop- eration rates positive, reaching a maximum for intermediate benefit-to-cost ratios (Fig. 44). If the benefit-to-cost ratio is too low, we find that each of these leading-eight strategies can be invaded by ALLD, whereas if the ratio is too high, ALLC can invade (SI Appendix, Fig. S5). In between, consistent standing may outperform ALLC and ALLD, but in the presence of noise it does not yield high cooperation rates against itself. Even if all interactions are observed (q = I), cooperation rates in a homoge- neous L2 population drop below 70% once the error rate exceeds 5% (SI Appendix, Fig. S4). Our analytical results in SI Appendix suggest that while L2 populations always recover from single dis- agreements, it may take them a substantial time to do so, during which further errors may accumulate. As a result, whereas L2 seems most robust when coevolving with ALLC and ALLD, it is unable to maintain full cooperation. Furthermore, additional simulation results suggest that even if L.2 is able to resist invasion by ALLC and ALLD, it may be invaded by mutant strategies that differ in only one bit from L2 (SI Appendix, Fig. S6). So far, we have assumed that mutations are rare, such that populations are typically homogeneous. Experimental evidence, however, suggests that there is considerable variation in the social norms used by subjects (4, 7-11). While some subjects are best classified as unconditional defectors, others act as uncon- ditional cooperators or use more sophisticated higher-order strategies (I I). In agreement with these experimental studies, there is theoretical evidence that some leading-eight strategies like L7 may form stable coexistences with ALLC (36). In SI Appendix, Figs. S7-59, we present further evolutionary results for higher mutation rates, in which such coexistences are possible. 0 LI A 1.0 e 0.8 10.6 0.4 • 3 0 ' Os 0.0 1 • L 1.3 2 a L4 L5 16 9 9 $ 3 5 7 Benefit b 9 B 9 Ci L7 18 • 0 8 t S 9 0 0.0 0 0_ 0 0 0 0 0 0.01 0.05 0.09 0.1 0.3 0.5 0.7 0.9 Error probstaity e Observation probability ct Flg. 4. Noise can prevent the evolution of full cooperation even if leading- eight strategies evolve. We repeated the evolutionary simulations in Fig. 3, but varying (A) the benefit of cooperation. (8) the error rate, and (C) the observation probability. The graph shows the average cooperation rate for each scenario in the selection-mutation equilibrium. This cooperation rate depends on how abundant each strategy is in equilibrium and on how much cooperation each strategy yields against itself in the presence of noise. For five of the eight scenarios, cooperation rates remain low across the con- sidered parameter range. Only the three other leading-eight strategies can persist in the population, but even then cooperation rates typically remain below 70%. We use the same baseline parameters as in Fig. 3. Oaf 6 I www.pnas.orgrcgi/dol/10.10734mas.0310565115 MI6e et al. EFTA00803981 1 Them we show that in the three cases LI, L2, and L7, popula- tions may consist of a mixture of the leading-eight strategy and ALLC for a considerable time. However, in agreement with our ram-mutation results, we find for LI and L7 that this mixture of leading-eight strategy and ALLC is susceptible to stochastic invasion by ALLD. Discussion Indirect reciprocity explores how cooperation can be maintained when individuals assess and act on each other's reputations. Sim- ple strategies of indirect reciprocity like image scoring (21, 22) have been suspected to be unstable, because players may abstain from punishing defectors to maintain their own good score (27). In contrast, the leading-eight strategies additionally take the con- text of an interaction into account. They have been considered to be prime candidates for stable norms that maintain coop- eration (20, 31). Corresponding models, however, assume that each pairwise interaction is witnessed only by one observer, who disseminates the outcome of the interaction to all other popula- tion members. As a consequence, the resulting opinions within a population will be perfectly synchronized. Even if donors are subject to implementation errors, or if the observer misperceives an interaction, all players will have the same image of the donor after the interaction has taken place. While the assumption of perfectly synchronized reputations is a useful idealization, we believe that it may be too strict in some applications. Subjects often differ in the prior information they have, and even if everyone has access to the same information [as is often the case in online platforms (51,52)], individuals differ in how much weight they attribute to different pieces of evidence. As a result, individuals might disagree on each other's reputa- tions. These disagreements can proliferate over time. Herein, we have thus systematically compared the performance of the leading-eight strategies when information is incomplete, private, and noisy. The leading-eight strategies differ in how they are affected by the noise introduced by private perception errors. Strategies like stem judging, that have been shown to be highly successful under public information (18, 32, 33), fail to distin- guish between friend and foe when information is private. While we have considered well-mixed populations in which all play- ers are connected, this effect might be even more pronounced when games take place on a network (53, 54). If players are able only to observe interactions between players in their immediate neighborhood, network-structured populations may amplify the problem of incomplete information. Pairwise interactions that one player is able to observe may be systematically hidden from his neighbor's view. Thus, the study of indirect reciprocity on networks points to an interesting direction for future research. The individuals in our model are completely independent when forming their beliefs. In particular, they are not affected by the opinions of others, swayed by gossip and rumors, or engaged in communication. Experimental evidence suggests that even when all subjects witness the same social interaction, gos- sip can greatly modify beliefs and align the subjects' subsequent behaviors (13). Seen from this angle, our study highlights the importance of coordination and communication for the stability of indirect reciprocity. Social norms that fail when information is noisy and private may sustain full cooperation when information is mutually shared and discussed. Materials and Methods Model Setup. We consider N individuals in a well-mixed population. Each player's strategy is given by a pair (a, a The first component I. Alexander R (1987) The &obey of Moral sntems (Aldine de Cayter, New Toth). 2. Rand DG, Nowak MA (2012) Human cooperation. Trends Cogn Ski 117:413-425. 3. Malls AP, Semmann D (2010) How Is human cooperation different? PAWS Trans A Soc 8 365:2663-2674. a =(npcp• ago. a Kg. naa, nen. a 0 b. ooOsp nom). corresponds to the player's assessment rule. An entry nµy is equal to one if the player assigns a good reputation to a donor of reputation x who chooses action A against a recipient with reputation y. Otherwise, if such a donor is considered as bad, the corresponding entry is zero. The second component of the strategy, = 099. .9,0, /So, 4 4,), (2) gives the player's action rule. An entry 9y is equal to one if the focal player with reputation x cooperates with a recipient with reputation y; otherwise it is zero. The assessment and action rules of the leading-eight strategies are shown in Table 1. We define ALLC as the strategy with assessment rule a = (I 1) and action rule $ = (1 1). ALLD is the strategy with n = (0 0) and =(0 0). Reputation Dynamks. To simulate the reputation dynamics for players with fixed strategies, we consider the image matrix (34-36) Mit) = (NO) of a population at time t. Its entries satisfy mii(t)= 1 if player i deems player j as good at time t and mg(t)=0 otherwise. We assume that initially, all players have a good reputation, rev(0)= 1 for all 1, j. However, our results are unchanged if the players' initial reputations are assigned randomly. We get only slightly different results if all initial reputations are bad; in that case, L7 players are unable to acquire a good reputation over the course of the game (for details, see SI Appendix). In each round t, two players i and j are drawn from the population at ran- dom, a donor and a recipient. The donor then decides whether to cooperate. Her choice is uniquely determined by her action rule y9 and by the reputations she assigns to herself and to the recipient, me(t)and :7),(0. The donor andthe recipient alwaysobservethedonor's decision; all other players independently observe it with probability q. With probability e, a player who observes the donor's action misperceives it, independent of the other players. All players who observe the interaction update their assessment of the donor according to their assessment rule. This yields the image matrix M(t + 1). We iterate the above elementary process over many rounds (our num- bers are based on 106 rounds or more). Based on these simulations, we can now calculate how often player i considers j to be good on average and how often player i cooperates with j on average. If the estimated painvise cooperation rate of i against j is given by we define player i's payoff as = I S s f; fxtP —cup. Evolutionary Dynamks. On a larger timescale, we assume that players can change their strategies (n, .3). To model the strategy dynamics, we consider a pairwise comparison process (39-41). In each time step of this process, one individual is randomly chosen from the population. With probability this individual then adopts a random strategy, with all other available strategies having the same probability to be picked. With the remaining probability 1 -;a the focal individual i chooses a random role model j from the population. If the players' payoffs are *; and cu, player i adopts fs strategy with probability P(*i. fra = (1 + exn( -Art; - 100' (SS). The parameters > 0 is the "strength of selection." It measures how strongly imi- tation events are biased in favor of strategies with higher payoffs. For s = 0 we obtain P(ti, = 1/2, and imitation occurs at random. Ass increases, payoffs become increasingly relevant when i considers imitating Ps strategy. In the main text, we assume players can choose only between a leading- eight strategy L,, ALLC, and ALLO. As we show in SI Appendix, Fig. 56, the stability of a leading-eight strategy may be further undermined if additional mutant strategies are available. Moreover, in the main text we report only results when mutations are comparably rare (43, 44). In SI Appendix, Figs. 57-59 we show further results for substantial mutation rates. Given the players' payoffs for each possible population composition, the selection- mutation equilibrium can be calculated explicitly. All details are provided in St Appendix. ACKNOWLEDGMENTS. This work was supported by the European Research Council Start Grant 279307 Graph Games (to K.C.), Austrian Science Fund (FWF) Gram P23499-N23 (to K.C.), FWF Nationale Forschungsnetzerke Grant S110074123 Rigorous Systems Engineering/Systematic Methods in Systems Engineering (to K.C.), Office of Naval Research Grant N00014-16-1-2914 (to MAN.), and the John Templeton Foundation (M.A.N.). C.H. adcnowledges generous support from the ISTFELLOW program. 4. Engelman D, Fighbacher U (2009) Indirect reciprocity and strategic reputation building in an experimental helping game. Games (con &hate 67:399-407. 5. Mcquet ), Revert C, Traulsen A, Milins1/41 M (2011) Shame and honour drive cooper- ation. Biel Lett 7:899-901. Hilbe et el. 9NAS Latest Articles I 5 of 6 EFTA00803982 6. Ohtsuki µ Masa Y, Nowak MA (2015) Reputation effects in public and private InteraCtiOns. PIPS COmputfllei 11:0004527. 7. Wedekind C. MBlnskl M (2000) Cooperation through image sowing In humans. Science 288450-852. 8. Seinen I. Schram A 2806) Social status and group norms: Indirect reciprocity In a repeated helping experiment. Fur &on Rev SO:581-602. 9. Bolton G, Kaiak E, Ockenfeb A (2005) Cooperation among strangers with limited Information about reputation. / Public Eton 89:1957-1468. 10. van Apeldoorn 1, Sduam A (2016) Indirect reciprocity; a field experiment. PEoS One 11:e0152076. it. 5walunan V, Molleman L Ule A Egos M (2016) Reputation-based cooperation: Empirical evidence for behavioral strategies. Evof Num &hay 37230-235. 12. Capron) V. Giardini F. Vilone D. Padua, M (2016) Partner selection supported by opaque reputation promotes cooperative behavior. Judgment Deck Making 11:589- 600. 13. Sommerfeld RD. Krambedc Hl. Semmann D, Milinski M (2007) Gossip as an alterna- tive for direct observation in games of indirect reciprocity. Proc Nall Arad Sri LISA 109:17435-17490. 19. Nowak MA. Sigmund K 2005) Evolution of indirect reciprocity. Nature 937:1291- 1298. 15. Nowak MA (2006) Five rule. har the evolution of cooperation. Science 319:1560-1563. 16. Sigmund K (2010) The Calculus of Selfishness (Princeton Univ Press, Princeton). 17. Sigmund K (2012) Moral assessment in indirect reciprocity./ Thew Bid 29925-30. 18. Santos FP. SantOS FC, Pache<0 1M (2018) SOCHI norm complexity and past reputations in the evolution of cooperation. Nature 555242-245. 19. Brandt H. Sigmund K (2004) The logic of reprobation: Assessment and action rules for Indirect reciprocation. Theor Blot 231:475-486. 20. Ohtwki µ Iwasa Y (2004) How should we define goodness?—Reputation dynamics In indirect reciprocity. Theo, VW 231:107-120. 21. Nowak MA, Sigmund K (1998) Evolution of indirect reciprocity by image scoring. Nature 393:573-577. 22. Nowak MA Sigmund K (1998) The dynamics of indirect reciprocity / flyer Blot 199:561-5M 23. 0Mwkill (2004) Reactive strategies In indirect reciprocity./ Theor Riot 227:299-314. 24. Brandt H. Sigmund K (2005) Indirect reciprocity image scoring, and moral hazard. Prue Nati Aced Sd USA 1022666-2670. 25. Berger U (2011) Learning to cooperate via indirect reciprocity. Games Eton 'they 72:30-37. 24. Berger U, Gr0ne A (2016) On the stability of cooperation under indirect reciprocity with first-order information. Games Eton 8ehav 9819-33. 27. Leimar 0, Hammerstein P (2001) Evolution of cooperation through indirect reci- procity. Prot R Sot 8 268745-753. 28. Pandianathan K. Boyd R (2003) A tale of two defectors: The Importance of standing for evolution of indirect reciprocity. f Theor Biol 229:11S-126. 29. Suzuki S. Akiyama E (2007) Evolution of indirect reciprocity in groups of various sizes and comparison with direct reciprocity. Theor Siof 24S:539-S52. 30. Sugden R (1986) The Economics of Rights. Co-Operation and Welfare (Blackwell Oxford). 31. Ohtwki µ hvasa Y (2006) The leading eight: Social norms that can maintain cooperation by indirect reciprocity./ mew Rio? 239:43S-444. 32. Osalub FACC. Santos FC, Pacheco 1M (2006) The evolution of norms. 1 Theor Rio! 241:233-240. 33. Santos FP, Santos FC. Padteco 116 2016) Social norms of cooperation In small-scale societies. PLoS Compost Bid 12:e1004709. 34. Uchida S (2010) Effect of private information on indirect reciprocity. Phys Rev E 82016111. 35. Uchida 5, Sasaki 7 (2013) Effect of assessment error and private information on stem- judging in indirect reciprocity. MOSSO/irons Tract.* 56:175-180. 36. Okada I, Sasaki T, Nakai Y (2017) Tolerant indirect reciprocity can boost social wel- fare through solidarity with unconditional cooperators in private monitoring. Sc) Rep 7.9737. 37. Okada I, Sasaki T, Nakai Y (2018) A solution for private assessment in indirect reciprocity using solitary observation. / Thew Blot 455:7-15. 38. Martinez-Vaquero IA Cuesta M (2013) Evolutionary stability and resistance to cheating in an indirect reciprocity model based on reputation. Phys Rev E 87: 052810. 39. Traulsen A Nowak MA Pacheco 1M (2006) Stochastic dynamics of invasion and fixation. Phys Rev E 74.011909. 40. Stewart Al. Plotkin 18 (2013) From extortion t0 generosity evolution In the iterated prisoner's dilemma. Proc Nati Aced Sri USA 110:15348-15353. 41. Reiter 1G, Hilbe C. Rand DG, Chatterlee K. Nowak MA 2018) Crosstalk in concurrent repeated games impedeseked reciprocityand requires stronger levels of forgiveness. Nat Common 9:555. 42. Mb. C, limsa S. Chatterjee K. Nowak MA (2018) Evolution of cooperation In stochastic games. Nature 559246-249. 43. Fudenberg D, ImhOf LA (2006) Imitation processes with small mutations, /EWA Thera' 131:251-262. 44. Wu B, Gokhale CS, Wang L Traulsen A (2012) How small are small mutation rates? 1 Math B/0160:803-827. 45. Sasaki T, Okada I, Nakai Y (2017) The evolution of conditional moral assessment in indirect reciprocity. Sri Rep 7:91870. 46. Hofbauer 1, Sigmund K (1998) EvOlut/Onary Games and Population Dynamics (Cambridge Univ Press, Cambridge, UK). 47. Szolnoki A. Szabo G (2004) Phase transitions for rock-scissorypaper game on different networks. Mys Rev E 70037102 48. Claussen 1C, Traulsen A (2008) Cyclic dominance and biodiversity it well-mixed populations, Phys ROV Lett 100:058104. 09. Szolnoki A, et al. (2014) Cyclic dominance in evolutionary games: A review. J R Soc Interface 11:20140735. 50. Stewart Al, Parsons 'Ti.. Plotkin 18 (2016) Evolutionary consequences of behavioral diversity. hoc Nail Arad Sri USA 111:67003-87009. 51. Resnick P. Zeckhauser R. Swanson 1. Lockwood K (2006) The value of reputation on 'Bay: A controlled experiment. Exp Econ 9:79-101. 52. Restive M, van de Rip A (2012) Experimental study of informal rewards in peer priakidiOn. P105 One 7:e34358. 53. Lieberman E, Hauert C, Nowak MA (2005) Evolutionary dynamics on graphs. Nature 933:112-316. 54. Szab0 G, Path G (2007) Evolutionary games On graphs. PhYS Rep 946.97-216. SS. Szabo G, Take C (1998) Evolutionary prisoner's dilemma game on a square lattice. Phys Rev E 5849-73. 6 of 6 I www.pnat.Org/Cgifdel/10.1073RanaS.1810565115 Mb* et al. EFTA00803983