Skip to main content
Skip to content
Case File
efta-efta01146514DOJ Data Set 9Other

LETTER

Date
Unknown
Source
DOJ Data Set 9
Reference
efta-efta01146514
Pages
54
Persons
0
Integrity

Summary

Ask AI About This Document

0Share
PostReddit

Extracted Text (OCR)

EFTA Disclosure
Text extracted via OCR from the original document. May contain errors from the scanning process.
LETTER doi:10.3.038/nature11467 Spontaneous giving and calculated greed David G. Randt•2-', Joshua D. Greene2+ & Martin A. Nowakm•ss Cooperation is central to human social behaviour". However, choosing to cooperate requires individuals to incur a personal cost to benefit others. Here we explore the cognitive basis of cooperative decision-making in humans using a dual-process framework"-". We ask whether people are predisposed towards selfishness, behav- ing cooperatively only through active self-control; or whether they are intuitively cooperative, with reflection and prospective reason- ing favouring 'rational' self-Interest. To investigate this issue, we perform ten studies using economic games. We find that across a range of experimental designs, subjects who reach their decisions more quiddy are more cooperative. Furthermore, forcing subjects to decide quickly increases contributions, whereas instructing them to reflect and forcing them to decide slowly decreases con- tributions. Finally, an induction that primes subjects to trust their intuitions increases contributions compared with an induction that promotes greater reflection. To explain these results, we propose that cooperation is intuitive because cooperative heuristics are developed in daily life where cooperation is typically advantageous. We then validate predictions generated by this proposed mechanism. Our results provide convergent evidence that intuition supports coopera- tion in social dilemmas, and that reflection can undermine these cooperative Impulses. Many people are willing to make sacrifices for the common good". Here we explore the cognitive mechanisms underlying this cooperative behaviour. We use a dual-process framework in which intuition and reflection interact to produce decisions'°-".". Intuition is often associated with parallel processing, automaticity, effortlessness, lack of insight into the decision process and emotional influence. Reflection is often associated with serial processing, effortfulness and the rejection of emotional influence"-1".". In addition, one of the psychological features most widely used to distinguish intuition from reflection is processing speed: intuitive responses are relatively fast, whereas reflective responses require additional time for deliberation". Here we focus our attention on this particular dimension, which is closely related to the distinction between automatic and controlled processing". Viewing cooperation from a dual-process perspective raises the following questions are we intuitively self-interested, and is it only through reflection that we reject our selfish impulses and force ourselves to cooperate? Or are we intuitively cooperative, with reflection upon the logic of self-interest causing us to rein in our cooperative urges and instead act selfishly? Or, alternatively, is there no cognitive conflict between intuition and reflection? Here we address these questions using economic cooperation games. We begin by examining subjects' decision times. The hypothesis that self-interest is intuitive, with prosociality requiring reflection to override one's selfish impulses, predicts that faster decisions will be less cooperative. Conversely, the hypothesis that intuition preferentially supports prosocial behaviour, whereas reflection leads to increased selfishness, predicts that faster decisions will be more cooperative. As a first test of these competing hypotheses, we conducted a one- shot public goods game" (PGG) with groups of four participants. I 8 0 We recruited 212 subjects from around the world using the online labour market Amazon Mechanical Turk (AMT)". AMT provides a reliable subject pool that is more diverse than a typical sample of college undergraduates (see Supplementary Information, section 1). In accordance with standard AMT wages, each subject was given USS0.40 and was asked to choose how much to contribute to a common pool. Any money contributed was doubled and split evenly among the four group members (see Supplementary Information, section 3, for experimental details). Figure la shows the fraction of the endowment contributed in the slower half of decisions compared to the faster half. Faster decisions result in substantially higher contributions compared with slower decisions (rank sum test, P= 0.007). Furthermore, as shown in Fig. lb, we see a consistent decrease in contribution amount with a 75% 65% - 55% - 45% - 35% b 100% 80% 60% 40% 20% 0% I I Faster decisions Sower decisions 1-10s >10s 4 56 12 I6 4 02 0.6 1 1.4 1.8 2.2 Decision tine (loa,,N) Figure 1 I Faster decisions are more cooperative Subjects who reach their decisions more quickly contribute more in a one-shot PCG (n = 212). This suggests that the intuitive response is to be cooperative. a, Using a median split on decision time, we compare the contribution levels of the faster half versus slower half of decisions. The average contribution is substantially higher for the faster decisions. b, Plotting contribution as a function of 168w- transformed decision time shows a negative relationship between decision time and contribution. Dot size is proportional to the number of observations, listed next to each dot. Error bars, mean = s.e.m. (see Supplementary Information, sections 2 and 3, for statistical analysis and further details). 'Program kw Evolutionary Dynarnics.Karrard Univers4y.Cambridge.Massachusells02138.USA.2Deperlma4 ol Psycholori.HanrardUniversty.Combdidge.Abssachuselts02138.USA.30eparlmentor Psychdogy.YaleUroversily.NewHaren.Connecleul 0652Q USA 'Deportment ol lAathanalics. Harvard LInnerstly.Carribridge.Massachuselts02138.USODepartnenl ol Organismic and Evolutionary Bool3gy. Harvard Unwersrty.Cambndge.i.lassechusetts0213a USA •Theseauthen ccobibuled equally to the work SEPTEMBER 2012 I VOL 489 I NATURE I 4 2 7 02012 Macmillan Publishers Limited. All rights reserved EFTA01146514 RESEARCH LETTER increasing decision time (Tobit regression, coefficient = -15.84, P=0.019; see Supplementary Information, sections 2 and 3, for statistical details). These findings suggest that intuitive responses are more cooperative. Next we examined data from all of our previously published social dilemma experiments for which decision time data were recorder-22. In these studies, conducted in the physical laboratory with college students, the experimental software automatically recorded decision times, but these data had not been previously analysed. To examine the psychology that subjects bring with them into the laboratory, we focused on play in the first round of each experimental session. In a one-shot prisoner's dilemma (ri = 48)20, a repeated prisoner's dilemma with execution errors (it = 278)31, a repeated prisoner's dilemma with and without costly punishment (pi = 104)", and a repeated PGG with and without reward and/or punishment (n = 192)7, we find the same negative relationship between decision time and cooperation (see Supplementary Information, section 4, for details). These results show the robustness of our decision-time findings: across a range of experi- mental designs, and with students in the physical laboratory as well as with an international online sample, faster decisions are associated with more prosociality. We now demonstrate the causal link between intuition and coop- eration suggested by these correlational studies. To do so, we recruited another 680 subjects on AMT and experimentally manipulated their decision times in the same one-shot PGG used above. In the 'time pressure' condition, subjects were forced to reach their decision quickly (within 10 s). Subjects in this condition have less time to reflect than in a standard PGG, and therefore their decisions are expected to be more intuitive. In the 'time delay' condition, subjects were instructed to carefully consider their decision and forced to wait for at least lOs before choosing a contribution amount. Thus, in this condition, decisions are expected to be driven more by reflection (see Supplementary Information, section 5, for experimental details). The results (Fig. 2a) are consistent with the correlational observa- tions in Fig 1. Subjects in the time-pressure condition contribute sig- nificantly more money on average than subjects in the time-delay condition (rank sum, P< 0.001). Moreover, we find that both manip- ulation conditions differ from the average behaviour in the baseline experiment in Fig. 1, and in the expected directions: subjects under time-pressure contribute more than unconstrained subjects (rank sum, P= 0.058), whereas subjects who are instructed to reflect and delay their decision contribute less than unconstrained subjects (rank sum, P = 0.028), although the former difference is only marginally significant. See Supplementary Information, section 5, for regression analyses. Additionally, we recruited 211 Boston-area college students and replicated our time-constraint experiment in the physical laboratory with tenfold higher stakes (Fig. 2b). We find again that subjects in the time-pressure condition contribute significantly more money than subjects in the time-delay condition (rank sum, P= 0.032). We also assessed subjects' expectations about the behaviour of others in their group, and find no significant difference across conditions (rank sum, P= 0.360). Thus, subjects forced to respond more intuitively seem to have more prosocial preferences, rather than simply contributing more because they are more optimistic about the behaviour of others (see Supplementary Information, section 6, for experimental details and analysis). We next used a conceptual priming manipulation that explicitly invokes intuition and reflection'. We recruited 343 subjects on AMT to participate in a one-shot PGG experiment. The first condition promotes intuition relative to reflection: before reading the PGG instructions, subjects were assigned to write a paragraph about a situ- ation in which either their intuition had led them in the right direction, or careful reasoning had led them in the wrong direction. Conversely, the second condition promotes reflection: subjects were asked to write about either a situation in which intuition had led them in the wrong S O a 75% 65% 55% S 45% 35% b 75% 65%- 55% - S O 45% 35% O 75% 65% - 55% - Time pressure 45% - 35% I I I Unconstrained Time delay Constraint condition Contribution Prediction of others' contribution Time pressure Time delay Constraint condition Promote intuition or Promote reflection or inhibit reflection inhibit intuition Priming conckhon Figure 2 I Inducing intuitive thinking promotes cooperation. a. Forcing subjects to decide quickly (10 s or less) results in higher contributions, whereas forcing subjects to decide slowly (more than 10 s) decreases contributions (n = 680). This demonstrates the causal link between decision time and cooperation suggested by the correlation shown in Fig. 1. b, We replicate the finding that forcing subjects to decide quickly promotes cooperation in a second study run in the physical laboratory with tenfold larger stakes = 211). We also find that the time constraint has no significant effect on subjects' predictions concerning the average contributions of other group members. Thus, the manipulation acts through preferences rather than beliefs. c. Priming intuition (or inhibiting reflection) increases cooperation relative to priming reflection (or inhibiting intuition) (n = 343). 'This finding prosides further evidence for the specific role of intuition versus reflection in motivating cooperation. as suggested by the decision time studies. Error bars, mean ± s.c.m. (see Supplementary Information, sections 5-7, for statistical analysis and further details). direction, or careful reasoning had led them in the right direction. Consistent with the seven experiments described above, we find that contributions are significantly higher when subjects are primed to promote intuition relative to reflection (Fig. 2c; rank sum, P= 0.011; see Supplementary Information, section 8, for experimental details and analysis). These results therefore raise the question of why people are intuitively predisposed towards cooperation. We propose the follow- ing mechanism: people develop their intuitions in the context of daily life, where cooperation is typically advantageous because many important interactions are repeated""22, reputation is often at 428 1 NATURE 1 VOL 489 20 SEPTEMBER 201: 02012 Macmillan Publishers Limited. All rights reserved EFTA01146515 LETTER RESEARCH stake's" and sanctions for good or bad behaviour might exist9b Thus, our subjects develop cooperative intuitions for social interactions and bring these cooperative intuitions with them into the laboratory. As a result, their automatic first response is to be cooperative. It then requires reflection to overcome this cooperative impulse and instead adapt to the unusual situation created in these experiments, in which cooperation is not advantageous. This hypothesis makes clear predictions about individual difference moderators of the effect of intuition on cooperation, two of which we now test. First, if the effects described above result from intuitions formed through ordinary experience, then greater familiarity with laboratory cooperation experiments should attenuate these effects. We test this prediction on AMT with a replication of our conceptual priming experiment. As predicted, we fmd a significant interaction between prime and experience it is only among subjects naive to the experimental task that promoting intuition increases cooperation (Fig. 3a; see Supplementary Information, section 9, for experimental details and statistical analysis). This mechanism also predicts that subjects will only fmd coopera- tion intuitive if they developed their intuitions in daily-life settings in which cooperation was advantageous. Even in the presence of repe- tition, reputation and sanctions, cooperation will only be favoured if enough other people are similarly cooperative". We tested this pre- diction on AMT with a replication of our baseline correlational study. As predicted, it is only among subjects that report having mainly cooperative daily-life interaction partners that faster decisions are a 75% b I 65% 55% 45% 35% 75%. 65% - 55%. 45% - 35% Naive ■ Primed to promote intuition • Primed to promote reflection I _I_ Experienced Previous experience with experimental setting ■ Faster dectsicns ■ Slower decisions Cooperative Uncooperative Opinion of daily-life interaction partners Figure 3 I Evidence that cooperative intuitions from daily lift spill over into the laboratory. Two experiments validate predictions of our hypothesis that subjects develop their cooperative intuitions in the context ofdaily life, in which cooperation is advantageous. a, Priming that promotes reliance on intuition increases cooperation relative to priming promoting reflection,but only among naive subjects that report no previous experience with the experimental setting where cooperation is disadvantageous (ra = 256). b, Faster decisions arc associated with higher contribution levels, but only among subjects who report having cooperative daily-life interaction partners = 341). As in Fig. la, a median split is carried out on decision times, separating decisions into the faster versus slower half. Error bars, mean ± s.e.m. (see Supplementary Information• sections 9 and 10. for statistical analysis and further details). associated with higher contributions (Fig. 3b; see Supplementary Information, section 10, for experimental details and statistical analysis). Thus, there are some people for whom the intuitive response is more cooperative and the reflective response is less cooperative; and there are other people for whom both the intuitive and reflective responses lead to relatively little cooperation. But we find no cases in which the intuitive response is reliably less cooperative than the reflective res- ponse. As a result, on average, intuition promotes cooperation relative to reflection in our experiments. By showing that people do not have a single consistent set of social preferences, our results highlight the need for more cognitively com- plex economic and evolutionary models of cooperation, along the lines of recent models for non-social decision-making'ra'-16. Furthermore, our results suggest a special role for intuition in promoting coopera- tion". For further discussion, and a discussion of previous work exploring behaviour in economic games from a dual-process perspec- tive, see Supplementary Information, sections 12 and 13. On the basis of our results, it may be tempting to conclude that cooperation is 'innate' and genetically hardwired, rather than the product of cultural transmission. This is not necessarily the case: intuitive responses could also be shaped by cultural evolution" and social learning over the course of development. However, our results are consistent with work demonstrating spontaneous helping behaviour in young children". Exploring the role of intuition and reflection in cooperation among children, as well as cross-culturally, can shed further light on this issue. Here we have explored the cognitive underpinnings of cooperation in humans. Our results help to explain the origins of cooperative behaviour, and have implications for the design of institutions that aim to promote cooperation. Encouraging decision-makers to be maximally rational may have the unintended side-effect of making them more selfish. Furthermore, rational arguments about the import- ance of cooperating may paradoxically have a similar effect, whereas interventions targeting prosocial intuitions may be more successful30. Exploring the implications of our findings, both for scientific under- standing and public policy, is an important direction for future study: although the cold logic of self-interest is seductive, our first impulse is to cooperate. METHODS SUMMARY Across studies 1.6, 8,9 and 10,a total of 1.955 subjects were recruited using AMR'" to participate in one of a series of variations on the one-shot PGG, played through an online survey website. Subjects received $030 for participating. and could earn up to SI more based on the PGG. In the PGG, subject were given S0A0 and chose how much to contribute to a 'common project'. All contributions were doubled and split equally among four group members. Once all subjects in the experiment had made their decisions, groups of four were randomly matched and the resulting payoffs were calculated. Each subject was then paid accordingly through the AMT payment system. and was informed about the average contribution of the other members of his or her group. No deception was used. In study 7, a total of 21I subjects were recruited from the Boston. Massachusetts. metropolitan area through the Harvard University Computer Laboratory for Experiment Research subject pool to participate in an experiment at the Harvard Decision Science Laboratory. Participation was restricted to students under 35 years of age. Subjects received a $5 show-up fee for arriving on time and had the opportunity to earn up to an additional S12 in the experiment. Subjects played a single one-shot PGG through the same website interface used in the AMT studies, but with tenfold larger stakes (maximum earnings of 510). Subjects were then asked to predict the average contribution of their other group numbers and had the chance to win up to an additional S2 based on their accuracy. These experiments were approved by the Harvard University Committee on the Use of Human Subjects in Research. For further details of the experimental meihods.set Supplementary Information. Received 13 December 2011; accepted 2 August 2012. 1. Trivers. R. The evolution of reciprocal altruism. Q. Rev. Biol. 46, 35-57 (1971) 12 SEPTEMBER 2012 I VOL 499 I NATURE 1 429 02012 Macmillan Publishers Limited. All rights reserved EFTA01146516 RESEARCH LETTER 2. Fudenberg.D.& Maskin. E. The folk theorem in repeated games with discounting or with incomplete information. Economefrica 54, 533-554 (1986} 3. Nowak M. A.& Sigmund. K Evolution of indirect reciprocity. Nature 437, 1291-1298(2005} 4. Boyd. R. Gintis. H.. Bowles. S.& Richerson. P. J. The evolution of altruistic punishment. Proc. Nat Acad. Sci. USA 100, 3531-3535 (2003). 5. Mil inski.M..Semrrenn. D.& KrambeckH.J. Reputation helps solve the 'tragedy of the commons. Nature 415.424-426 (2002). 6. Rockenbach. & Milinsld. M. The efficient interaction of indirect reciprocity and costly punishment. Nature 444, 718-723 (2006). 7. Rand, D.G.. Dreher, A.. Ellirgsen.T, Fudenberg. a& Nowak. M.A Positive interactions promote public cooperation. Science 325, 1272-1275 (2009). & Fehr, E& Gachter.S. Altruistic punishment in humans. Nature 415,137-140 (2002). 9. Rand, D. G..Arbesman. & & Christakis N. A. Dynamic social networks promote cooperation in experiments with humans. Proc Nay Aced Sci USA 108, 19193-19198 (2011). la Sloman, S.A. The empirkal case for two systems of reasonirg.Psychol Bull 119, 3-22 (1996). 11. Stanovich. K. E& West. R. F. Individual differences in rational thougbt. J. Exp. Psychal 127, 161-188(1998} 12 Chaiken.S.&Trope.Y. Dual-ProcessTheories in SocialPsycholog/(Gu Word, 1999} 13. Kahneman, D. A perspective on judgment and choice: mapping bounded rationality. Am. Psycho). 58, 697-720 (2003). 14. Plessner. Betsch. C. & Betsch. T. Intuition in Judgment and Decision Making (Lawrence Erlbaum. 2008} 15. Kahneman, D. Thinking Fast and Sfotv (Straus and Giroux. 2011} 16. Shiffrin.R_ M.& Schneider. W.Controlled and automatic information processing: II. Perceptual teaming, automatic attending, and a general theory. Psycho,. Rev. 84, 127-190(1977} 17. Milkw.E K&Ccben J. D.An integrative theory of prefrontal cortex function.Annu. Rev. Neumsci. 24,167-202 (2001). 18. Frederick S,Ccgnitive reflection and decision making.J.Econ.Parmect 19.25-42 (2005). 19. Horton. J. J., Rand, D.G. & Zeckhauser, R. J. The online laboratory: conducting experiments in a real labor market. Erp. Econ. 14, 399-425 (2011). 20. Pfeiffer. T.. Tran, L Krumme. C.& Rand. D. G. The value of reputation.J. R. Soc. Interface http://dxdoiorg/10.1098/rsif2012.0332 (20 June 2012). 21. Fudenberg. D.. Rand, a G.& Dreber.A Slow to anger and fast to forgive: cooperation in an uncertain world. Am. Econ. Rev. 102, 720-749 (2012} 22. Dreher. A, Rand, D.C.. Fudenberg.D.& Nowak M.A Winners don't punish. Nature 452, 348-351(2008). 23. Shenhay.A.. Rand. D. G. & Greene. J. D. Divine intuition: ccgnitive style influences belief in God. J. Exp. PsychoL Gen. 143.423-428 (2012). 24. Benhabib.J. & Bisin, A. Modeling internal commitment mechanisms and self- control: a neuroeconomics approach to consumption-saving decisions. Games Econ. BEAN. 52.460-492 (2005). 25. Fudenberg.D.& Levine. D. KA Dual-self model of impulse confrci.Am. Econ. Rev. 96, 1449-1476 (2C06} 2& McClure, S. M. Laibson. D. I, Loewenstein. G. & Cohen J. D. Separate neural systems value immediate and delayed monetary rewards. Science 306, 503-507 (2004). 27. Bowies S. & Gintis H. in The Economyas a EvolvingComplex System 3 (eds Blume, L and Durlaut S. N.) 339-364 (2002} 26 Richerson. P.J. & Boyd, R. Not byGenes Alone: How Culture Transformed Human Evolution. (Univ. Chicago Press 2005). 29. Wameken, F. & Tomasello. M. Altnistic helping in human infants and young chimpanzees. Science 311, 1301-1303 (2006). 30. Bowies S.Policies designed for seff-interested citizens mayundermine "the moral sentiments": evidence from economic experiments. Science 320, 1605-1609 f2003). Supplementary Information is available in the online version of the paper. Acknowledgements We thank H. Ahlblad, O. Amir. F. Fu. O. Hauser.). Horton and R Kane for assistance with carrying out the experiments. and P. Blake. S. BmvIes. N. Christakis. F. Cushman. A. Dreher. T. Ellingsen. F. Fu. D. Fudenberg. 0. Hauser. J. Jordan. M. Johannesson, M. Manapat J. Paxton. A Peysakhovkh. A Shenhay. J. Sirlin-Rand. M. van Veelen and 0. Wurzbacher for discussion and comments This work was supported in part bya National Science Foundation grant (SES-082197/3 to JD.G.). D.G.R and MAN. are supported by grants from the John Templeton Foundation. Author Contributions D.G.R..J.D.G. and MAN. designed the experiments D.G.R carried out the experiments and statistical analyses.ancl D.G.R.JD.G.and MAN. wrote the paper. Author Information Reprints and permissions information is available at bwnvnature.comireprints. The authors declare no competing financial interests Readers are welcome to comment on the online version of the paper. Correspondence and requests for materials should be addressed to D.G.R.([email protected]). 430 I NATURE I VOL 489 I 20 SEPTEMBER 2012 02012 Macmillan Publishers Limited. All nghts reserved EFTA01146517 SUPPLEMENTARY INFORMATION doi:10.1038/nature11467 1. Online recruitment procedure using Amazon Mechanical Turk 2 2. Log-transforming decision times 3 3. Study 1: Correlational decision time experiment on AMT 4 4. Studies 2 - 5: Reanalysis of previously published experiments run in the physical laboratory 6 5. Study 6: Time pressure / time delay experiment on AMT 12 6. Study 7: Time pressure / time delay experiment with belief elicitation in the physical laboratory 14 7. Behavior on AMT versus the physical laboratory (Study 6 vs Study 7) 17 8. Study 8: Conceptual priming experiment on AMT 18 9. Study 9: Conceptual priming experiment with experience measure and decision times on AMT 22 10. Study 10: Correlational experiment on AMT with moderators, individual differences in cognitive style, and additional controls 26 12. Implications for economic and evolutionary models 36 13. Previous dual-process research using economic games 37 14. Supplemental study: Experiment on AMT showing that detailed comprehension questions induce reflective thinking and reduce cooperation 38 15. Experimental instructions 40 References 47 WWW.NATURE-COM/NATUREI I EFTA01146518 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION 1. Online recruitment procedure using Amazon Mechanical Turk Subjects for many of the experiments in this paper were recruited using the online labor market Amazon Mechanical Turk (AMT)". AMT is an online labor market in which employers can employ workers to complete short tasks (generally less than 10 minutes) for relatively small amounts of money (generally less than $1). Workers receive a baseline payment and can be paid an additional bonus depending on their performance. This makes it easy to run incentivized experiments: the baseline payment is a `show-up fee,' and the bonus payment is determined by the points earned in the experiment. One major advantage of AMT is it allows experimenters to easily expand beyond the college student convenience samples typical of most economic game experiments. Among American subjects, AMT subjects have been shown to be significantly more nationally representative than college student sampled/. Furthermore, workers on AMT are from all around the world: in our experiments, 37% of the subjects lived outside of the United States, with more than half of the non-American subjects living in India. In our statistical analyses below, we show that there is no significant difference in the effects we are studying between US and non-US subjects. This diversity of subject pool participants is particularly helpful in the present study, given our focus on intuitive motivations that may vary based on life experience. Of course, issues exist when running experiments online that do not exist in the traditional laboratory. Running experiments online necessarily involves some loss of control, since the workers cannot be directly monitored as in the traditional lab; hence, experimenters cannot be certain that each observation is the result of a single person (as opposed to multiple people making joint decisions at the same computer), or that one person does not participate multiple times (although AMT goes to great lengths to try to prevent this, and we use filtering based on IP address to further reduce repeat play). Moreover, although the sample of subjects in AMT experiments is more diverse than samples using college undergraduates, we are obviously restricted to people that participate in online labor markets. To address these potential concerns, recent studies have explored the validity of data gathered using AMT (for an overview, see ref I). Most pertinent to our study are two quantitative direct replications using economic games. The first shows quantitative agreement in contribution behavior in a repeated public goods game between experiments conducted in the physical lab and those conducted using AMT with approximately 10-fold lower stakes2. The second replication again found quantitative agreement between the lab and AMT with 10-fold lower stakes, this time in cooperation in a one-shot Prisoner's Dilemmas. The latter study also conducted a survey on the extent to which subjects trust that they will be paid as described in the instructions (a critical element for economic game experiments) and found that AMT subjects were only slightly less trusting than subjects from a physical laboratory subject pool at Harvard University (trust of 5.4 vs 5.7 on a 7-point Likert scale). A third study compared behavior on AMT in games using $1 stakes with unincentivized games, examining the public goods game, the dictator game, the ultimatum game and the trust games. Consistent with previous research in the physical laboratory, adding stakes was only found to affect play in the dictator game, where subjects were significantly more generous in the unincentivized dictator game compared to the $1 dictator game. Furthermore, the average behavior in these games on AMT was within the range of WWW.NATURE.CONVNATURg 12 EFTA01146519 doi:10.1038/nature 11467 RESEARCH SUPPLEMENTARY INFORMATION averages reported from laboratory studies, demonstrating further quantitative agreement between AMT and the physical lab. In additional studies, it has also been shown that AMT subjects display a level of test-retest reliability similar to what is seen in the traditional lab on measures of political beliefs, self- esteem, Social Dominance Orientation, and Big-Five personality traits'', as well as belief in God, age, gender, education level and incomeI•6; and do not differ significantly from college undergraduates in terms of attentiveness or basic numeracy skills, as well as demonstrating similar effect sizes as undergraduates in tasks examining framing effects, the conjunction fallacy, and outcome bias7. The present studies add another piece of evidence for the validity of experiments run on AMT by comparing our AMT studies with decision time data from previous laboratory experiments (Main text Figure 2): Both online and in the lab, subjects that take longer to make their decisions are less cooperative. 2. Log-transforming decision limes In several of our experiments, we predict cooperation as a function of decision times. However, the distribution of decision times (measured in seconds) is heavily right-skewed, as we did not impose a maximum decision time (decision times for the baseline decision time experiment, Study 1, are shown in Figure S la). Thus linear regression is not appropriate using non- transformed decision times, as the few decision times that are extremely large exert undue influence on the fit of the regression. To address this issue, we log10-transform decision times in all analyses (log 10 transformed decision times for the baseline decision time experiment are shown in Figure S lb). As reported below, our main results are qualitatively similar if we instead analyze non-transformed decision times and exclude outliers (subjects with decision times more than 3 standard deviations above the mean decision time). a IL O O L 100 200 300 DeOeRN mme0ecaay b 4,2 LL O 1.5 WDOOSIOITim? jscenIsil 2.5 Figure SI. (a) Distribution of decision times in the baseline experiment. (b) Distribution of log10 transformed decision times in the baseline experiment. WWW.NATURE-COWNATURII3 EFTA01146520 doi:10.1038/nature 11467 RESEARCH SUPPLEMENTARY INFORMATION 3. Study 1: Correlational decision time experiment on AMT Methods In the baseline experiment (main text Figure I), subjects were recruited using AMT and told they would receive a $0.50 show-up fee for participating, and would have the chance to earn up to an additional $1.00 based on the outcome of the experiment. After accepting the task, subjects were redirected to website where they participated in the study. First subjects were shown the Instructions Screen, where they read a set of instructions describing the following one-shot public goods game: Players interacted in groups of 4; each player received 40 cents; players chose how many cents to contribute to the group (in increments of 2 to avoid fractional cent amounts) and how many to keep; all contributions were doubled and split equally by all group members. After they were finished reading the instructions, subjects clicked OK and were taken to the Contribution Screen. Here they entered their contribution decision and clicked OK. The website software recorded how long it took each subject to make her decision (in seconds), that is, the amount of time she spent on the Contribution Screen. Time spent on the Instructions Screen did not count towards our decision time measure. (Time spent on the Instructions Screen is examined below in Study 10 and shown not to influence cooperation.) After entering their contribution amount, subjects were taken to the Comprehension Screen in which they answered two comprehension questions to determine whether they understood the payoff structure: "What level of contribution earns the highest payoff for the group as a whole?" (correct answer = 40) and "What level of contribution earns the highest payoff for you personally?" (correct answer = 0). Subjects were then taken to a demographic questionnaire and given a completion code. We included comprehension questions after the contribution decision, rather than before as is typical in most laboratory experiments, because we were concerned about the possibility of pushing all of our subjects into a reflective mindset prior to their decision-making. (In SI Section 14, we discuss a supplemental experiment that validates this concern by demonstrating that subjects who complete comprehension questions, including a detailed payoff calculation, before making their decision choose to contribute significantly less than those who complete the comprehension questions afterward). Importantly, we show that our result is robust to controlling for comprehension, indicating that the negative relationship between decision time and cooperation is not driven by a lack of comprehension among the faster responders. Once the decisions of all subjects had been collected, subjects were randomly matched into groups of 4, payoffs were calculated, and bonuses were paid through AMT. Payoffs were determined exactly as described in the instructions, and no deception was used. WWW.NATURe.COM/NATUREI EFTA01146521 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION Results We begin with descriptive statistics: N=212 Mean Std Contribution 23.83 15.39 Decision time 15.92 22.96 Log I 0(Decision time) 1.03 0.34 Age 28.02 8.73 Gender (0=M, I=F) 0.42 0.49 US Residency (0=N, I=Y) 0.45 0.49 Failed Comprehension (0=N, 1=Y) 0.28 0.45 In the baseline experiment, we ask how the amount of time a subject takes to make her contribution decision relates to the amount contributed. To do so, we perform a set of Tobit regressions with robust standard errors, taking contribution amount as the dependent variable (Table SI). Tobit regression allows us to account for the fact that contribution amounts were censored at 0 and 40 (the minimum and maximum contribution amounts). In the first regression, we take log-10 transformed decision time as the independent variable, and find a significant negative relationship. In the second regression, we show that this effect remains significant when including controls for age, gender, US residency, and failing to correctly answering the comprehension questions, as well as dummies for education level. In the third regression, we show that this effect also remains significant when excluding extreme decision times for which there was comparatively little data (regression 3 includes only subjects with 0.6 < log10(decision time) < 1.2). We also continue to find a significant negative relationship between decision time and contribution (coeff=-0.497, p=0.018) using non-transformed decision times and excluding outliers (subjects with decision times more than 3 standard deviations above the mean [mean decision time = 15.9, std = 23.0 implies a cutoff of 85 seconds]) and including controls for age, gender, US residency and comprehension. It is worthwhile to note that the average level of contribution (59.6% of the endowment) of our subjects recruited from AMT is well within the range of average contribution levels observed in previous studies. Our PGG uses a marginal per capita return (MPCR) on public good investment of 0.5 (for every cent contributed, each player earns 0.5 cents). We used an MPCR of 0.5, rather than the value of 0.4 used in many previous studies (where contributions are multiplied by 1.6 and split amongst 4 group members), to create more easily divisible numbers and therefore simpler instructions for the AMT workers, many of whom are less sophisticated than university students. Previous lab studies that used an MPCR of 0.5 report average contribution levels of 40%-70%s"12, which are in line with our value of 59.6%. Thus our experiment adds to the growing body of literature demonstrating the validity of data gathered on AMT. WWW.NATURF-COWNATURI I EFTA01146522 doi:10.1038/nature 11467 RESEARCH SUPPLEMENTARY INFORMATION Table SI. PGG contribution regressed against decision time. (1) (2) (3) Decision time (logI0 seconds) -18.42** -15.84** -29.63** (7.285) (6.772) (15.06) US Residency (O=N, 1=Y) 2.829 2.210 (5.113) (5.666) Age 0.695 0.502 Gender (CM, 1=F) 0.402 2.598 (4.104) (4.794) Failed Comprehension ((-.N, 1=Y) -5.886 -8.789 (4.459) (5.306) Education dummies No Yes Yes Constant 49.01*** 25.91 25.21 (8.091) (22.99) (24.27) Observations 212 212 156 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 4. Studies 2 - 5: Reanalysis of previously published experiments run in the physical laboratory Here we analyze decision time data from all of our previously published cooperation experiments for which decision times were recorded1346. These experiments were conducted in the physical laboratory with Boston area college student participants, using the experimental software Ztreel I. We leverage the fact that Ztree automatically records decision times. Thus, although these experiments were not originally conducted to explore the role of intuitive versus reflective cognitive processes, that fact that we find the same negative relationship between cooperation and decision time found in our online experiments demonstrates the robustness of the effect to variations in experimental design, subject pool, and online versus physical laboratory recruitment/implementation. We note that unlike our AMT experiments, in these lab studies the subjects completed simple comprehension quizzes prior to beginning the experiment (with the exception of ref 16 which did not have a comprehension quiz). The details of these quizzes varied across experiments, but none involved the multiple detailed payoff calculations sometimes used in PGG laboratory experiments. Typical questions in our experiments, where subjects played Prisoner's Dilemma games, included "What is the probability of a subsequent round after round 1? After round 10?" or reading off entries in a Prisoner's Dilemma payoff matrix, such as "If you chose A and the other person chooses B, how many points do you get?" See SI Section 14 for a supplemental experiment exploring the effect of asking comprehension questions with detailed payoff calculations before versus after the contribution decision. WWW.NATURLCOM/NATUREI 6 EFTA01146523 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION We begin by analyzing the control treatment from ref 13 in which 48 subjects played a series of one- shot Prisoner's Dilemma games. In each interaction, subjects were randomly paired, and each pair simultaneously chose to either pay 10 units to give their partner 30 units (i.e. cooperate) or to do nothing (i.e. defect). After making a decision and being informed of their partner's decision, subjects were randomly rematched with new partners for another interaction. Players were given no information about their partner's play in previous games. In total, 29 such interactions occurred. We focus on the first decision subjects made in the experimental session (i.e. the first interaction). The first decision most cleanly represents the intuitions subjects bring into the laboratory by minimizing in-game learning, and also maximizes comparability to our one-shot experiments. Examining cooperation in the first interaction (using logistic regression with robust standard errors), we find a significant negative relationship between cooperation probability and decision time (coeff=-3.42, p4).014; Figure S2A). This relationship continues to exist (coeff=-3.37, p=0.062) when excludin decision times with relatively few observations (times less than 10°A seconds or more than 101. seconds). Using logistic regression with robust standard errors clustered on subject and session, we continue to find a significant effect (coeff=-0.95, p=0.047) when considering the first 5 interactions and controlling for interaction number, albeit with a smaller coefficient; but no longer find a significant effect when considering all 29 interactions (coeff=-0.03, p=0.931). Table S2. Cooperation in series of 1-shot PDs (data from Pfeiffer et aL (2012) J Royal Society Interface). Logistic regression. (1) (2) (3) (4) (5) Interaction 1 Ints 1-5 Ints 1-5 All Ints All Ints Decision time (logIO seconds) -3.417** -0.243 M.951** 0.268 -0.0261 (1.394) (0.432) (0.480) (0.306) (0.301) Interaction # -0.342*** -0.0542 (0.115) (0.0384) Constant 2.939** -0.370 1.092* -1.474*** -0.567 (1.308) (0.546) (0.632) (0.401) (0.639) Observations 48 240 240 1,392 1,392 Robust standard errors in paren heses *** <0.01, ** <0.05, * <0.1 When considering ref 13, we focus on the control condition described above because it demonstrates that our effect exists in one-shot games in the physical laboratory. The effect is not restricted, however, to the control condition. If we instead analyze the data from the 176 subjects that played a stochastically repeated indirect reciprocity game, we continue to find a negative relationship between decision time and cooperation. In these experiments, the setup is the same as the control, except that there is a reputation system such that after each PD, subjects' reputations are updated (to be either `good' or 'bad') based on an explicit assignment rule that is known to the subjects. There were three such conditions, with the assignment rule varying across conditions. Furthermore, subjects were allowed to buy and sell their reputations in two of the conditions. See ref 13 for more details. WWW.NATURS.COM/NATURE I] EFTA01146524 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION Examining cooperation in the first interaction (using logistic regression with robust standard errors), we find a significant negative relationship between cooperation probability and decision time (coeff=-1.85, p=0.002). This relationship continues to hold (coeff=-1.46, ps3.027) when including condition dummies. Using logistic regression with robust standard errors clustered on subject and session, we continue to find a significant effect when considering all 29 interactions (no controls: coeff=-1.31, p<0.001; controlling for round number and condition dummies: coeff = -0.96, p<0.001). Ref 13 also included a set of fixed-length game conditions that we do not reanalyze as the decision time data for those conditions are not available. Next we consider ref 15, where 278 subjects played a series of stochastically repeated 2-player Prisoner's Dilemma games with execution errors. In each round, there was a 1/8 probability of a player's move being switched to the opposite, and a 7/8 probability of a subsequent round occurring. The benefit-to-cost ratio of cooperation was varied across four different conditions, with b/c=[1.5, 2, 2.5 and 4). Examining cooperation in the first round of the first interaction (using logistic regression with robust standard errors and including condition dummies), we find a significant negative relationship between intended cooperation probability and decision time (coeff =-1.43, ps3.005, including controls for b/c ratio; Figure S2B). This relationship continues to exist (coeff=-1.15, ps3.053) when excluding decision times with relatively few observations (times of than 10 seconds). Moreover, we continue to find a significant effect when considering all decisions over the course of the session (standard errors clustered on subject and group, coeff=-0.97, p<0.001, including controls for b/c ratio, interaction number and round number), albeit with a smaller coefficient. Regressions are shown in Table S3. Table S3. Cooperation in stochastically repeated PD with execution errors (data from Fudenberg et al 2012 AER). Logistic regression. (1) (2) (4) (5) (6) 1st decision 1st decision All decisions All decisions All decisions Decision time (log10 seconds) -1.295*** -1.427*** -0.731*** M.777*** A.970*** (0.478) (0.504) (0.161) (0.119) (0.141) Interaction # 0.0199 (0.0124) Round # -0.187*** (0.0122) Condition dummies No Yes No Yes Yes Constant 0.937*** 1.342*** 0.132 0.459*** 1.296*** (0.222) (0.312) (0.156) (0.138) (0.189) Observations 278 278 26,584 26,584 26,584 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 Next we analyze ref 14, where 104 subjects played a series of stochastically repeated 2-player Prisoner's Dilemma games (without execution errors). After every round, there was a 3/4 probability WWW.NATURLCOM/NATUREI EFTA01146525 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION of a subsequent round. The benefit-to-cost ratio of cooperation and the availability of a 3nd option for costly punishment (pay 1 for the other to lose 4) were varied across treatments (4 treatments: low b/c without punishment, low b/c with punishment, high b/c without punishment, high b/c with punishment). Examining cooperation in the first round of the first interaction (using logistic regression with robust standard errors and including dummies for treatment), we again find a significant negative relationship between cooperation probability and decision time (coeff=-2.67, p=3.018; Figure S2C). This relationship continues to hold (coeff=-2.78, p".1.031) when excluding decision times with relatively few observations (times less than 10°A seconds or more than 101 seconds). Furthermore, we continue to find a significant relationship when analyzing all decisions over the course of the session (standard errors clustered on subject and group, coeff=-0.53, p=0.002), although the coefficient is smaller than in the first period. Regressions are shown in Table 54. Table S4. Cooperation in stochastically repeated PD w thAvithout costly punishment (data from Dreher et al 2008 Nature). Logistic regression. (1) (2) (4) (5) (6) 1st decision 1st decision All decisions All decisions All decisions Decision time (logI0 seconds) -2.741** -2.660** 0.254 -0.528*** -0.554*** (1.107) (1.123) (0.203) (0.171) (0.178) Interaction # -0.0128* (0.00752) Round # -0.361*** (0.0313) Condition dummies No Yes No Yes Yes Constant 2.887*** 2.522*** -0.275** 0.568*** 1.741*** (0.882) (0.961) (0.117) (0.210) (0.291) Observations 104 104 8,120 8,120 8,120 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 Finally, we consider ref 16, where 192 subjects played a repeated public goods game with persistent groups and identities. Subjects were given no information regarding the length of the game, which lasted 50 rounds. The possibility of targeted interaction was varied across four conditions: control PGG, PGG with costly punishment, PGG with costly reward, and PGG with both punishment & reward. As in our 1-shot PGG, tobit with robust standard errors find a significant negative correlation between first round contribution (0-20) and decision time (coeff=-26.38, p=0.001, including condition dummies; Figure S2D). This relationship continues to hold (coeff=-23.01, p=1006) when excluding decision times with relatively few observations (times less than 10°° seconds or more than 1012 seconds). WWW.NATURLCOM/NATUREI 9 EFTA01146526 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION The relationship between contribution and decision time, however, decays with experience: we find a significant effect when analyzing the first 10 periods (linear regression with standard errors clustered on subject and group, coeff=-3.26, p-A1.030), but not when analyzing periods 11 to 50 (linear regression with standard errors clustered on subject and group, coeff=-1.71, p3.275). We use linear regression rather than Tobit regression for the multi-round analyses as to our knowledge, the statistical software available to us cannot do multi-level clustering with Tobit regressions. Regressions are shown in Table S5. Table SS. Contribution in repeated PGG ivith/without targeted interactions (data from Rand et al 2009 Science). Note regressio is 1 and 2 use Tobit regression, while regression 3-6 use linear regression clustered on subject and group. (1) (2) (3) (4) (5) (6) Round 1 Round 1 Round 1-10 Round 1-10 Round 11-50 Round 11-50 Decision time (logI0 seconds) -25.92*** -26.38*** 3.424** 3.258** 1.63 -1.71 (7.430) (7.804) -1.403 -1.49 -1.769 -1.563 Condition dummies No Yes No Yes No Yes Constant 23.46*** 23.47*** I5.95*** I7.61*** I3.41*** I7.83*** -2.664 -3.249 -1.298 -1.359 -1.415 -1.627 Observations 192 192 1,920 1,920 7,680 7,680 R-squared - - 0.01 0.079 0.001 0.25 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 WWW .NATURE.COWNATURe 110 EFTA01146527 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION a 1-shot Prisoner's Dilemma b Repeated Prisoner's Dilemma with execution errors 1 1 0.9 . ..% 0.9 0.8 it. -. 0.8 . 56 o 0 o •7 ... 1:1 0.6 E & 0.5 8 0.4 0.2- 0.1- 111 .1.6 ..... 2 O 0.3 g 0.7 0.6 a 0.5 2 0.4 0 0.3 0.2- 01 0 i 0 C T 56 _ 113 ... i - lt,....... _ T 36 ..., - - -1- _ 9 - 1- - 2 1 5 r 02 0.4 0.6 0.8 1 12 1.4 02 0.4 0.6 0.8 1 12 14 Decision time (log10(sec]) Decision time (log1O[sec]) Repeated Prisoner's Dilemma with/without punishment 0.9 J 2 •-• 0.8 5 0.7 14 i 0.6 Zr) 0.5 8- 0.4 O O 0.3 0.2 0.1 0 110 d Contribution Repeated Public Goods Game with/without reward and/or punishment 25 20 15 10 5 0 t 1 2 33 02 0.4 0.6 0.8 1 12 1.4 0 02 0.4 0.6 0.8 1 1.2 1.4 Decision time (log1O[sec]) Decision time (log10(secll Figure S2. Reanalysis of previous experiments showing the first decision of the session in a series of I -shot Prisoner's Dilenunas" (a), a repeated Prisoner's Dilemma with execution errors" (b), a repeated Prisoner's Dilemma with or without costly punishment14 (c), and a repeated PGG with or without reward and/or punishment16 (d). Error bars indicate standard error of the mean. Dot size is proportional to number of observations, which are indicated next to each dot. WWW.NATURE.COWNATURE III EFTA01146528 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION 5. Study 6: Time pressure / time delay experiment on AMT Methods For Study 6, subjects were again recruited online using AMT. The experimental design was identical to that of the AMT correlational decision time experiment (Study 1), except that one additional piece of text was added to the screen on which subjects entered their PGG decision. In the `time pressure' condition, subjects were asked to make their decision as quickly as possible, and were informed that if they did not enter their decision within 10 seconds they would not be allowed to participate. In the 'time delay' condition, subjects were asked to think carefully about their decision before making it, and were informed that if they must wait at least 10 seconds before entering their decision or else they would not be allowed to participate. Subjects in the time pressure condition who took longer than 10 seconds were excluded, as were subjects in the time delay condition who took less than 10 seconds. However, the main result continues to hold even if these subjects are not excluded — see statistical analysis below. Results We begin with descriptive statistics: Subjects that obeyed time constraint All subjects Time pressure N= 94 Time delay N=249 Time pressure N=372 Time delay N=308 Mean Std Mean Std Mean Std Mean Std Contribution 26.98 14.06 20.88 14.42 23.31 14.65 21.49 14.57 Decision time 6.99 2.06 34.83 42.28 12.13 8.87 28.83 39.37 LoglO(Decision time) 0.82 0.15 1.44 0.26 1.00 0.26 1.29 0.37 Age 28.74 8.96 29.58 9.35 29.01 9.57 29.80 9.61 Gender (0=M, 1=F) 0.47 0.5 0.39 0.49 0.45 0.50 0.39 0.49 US Residency 0.57 0.5 0.43 0.5 0.46 0.50 0.41 0.49 Failed Comprehension 0.35 0.48 0.44 0.5 0.47 0.50 0.44 0.50 Disobeyed time constraint - - - - 0.48 0.50 0.19 0.39 In our time constrain experiment, we examine the effect of forcing subjects to make their decision in 10 seconds or less (the 'time pressure' condition) versus focusing them to stop and think for at least 10 seconds (the 'time delay' condition). To do so we perform a set of Tobit regressions with robust standard errors, taking contribution amount as the dependent variable (Table S6). Regression 1 shows that contributions were significantly lower in the time delay WWW.NATURE.COMMAIIIPSI II EFTA01146529 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION condition. Regression 2 shows that this continues to be true when controlling for age, gender, US residency, failing to correctly answering the comprehension questions and education. Regression 3 shows that this effect is robust to including subjects that disobeyed the time constraint. Table S6. Time pressure condition versus time delay condition. (1) (2) (3) Time pressure condition 10.91*** 10.59*** 5.535*** (2.474) (2.450) (2.022) US Residency (0=N, 1=Y) 4.500 3.805 (3.062) (2.451) Age 0.132 0.329 - - Gender (1".M, 1=F) 1.345 0.851 (2.529) (1.979) Failed comprehension -2.865 -0.694 (2.704) (2.140) Disobeyed time constraint -6.582*** (2.121) Education dummies No Yes Yes Constant 22.64*** -0.178 -0.839 (1.524) (8.588) (6.395) Observations 443 443 680 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 In addition to comparing the time pressure and time delay conditions to each other, we now compare both conditions to the baseline from Study 1 (while noting that behavior in the baseline varies substantially depending on reaction time, as per Table Si above). To do so, we conduct a set of Tobit regressions with robust standard errors on the data from Study 1 and Study 6 combined, creating two binary dummy variables: one indicating participation in the time pressure condition, and the other indicating participation in the time delay condition (Table S7). Regression 1 shows significantly lower contributions in the time delay condition compared to the baseline, and marginally significantly higher contributions in the time pressure condition compared to the baseline. Regression 2 shows that these relationships continue to hold when controlling for US residency, age, gender, failing to correctly answer the comprehension questions and education. Regression 3 shows that these relationships are robust to including subjects that did not obey the time constraint. WWW.NATURLCOMMAIIIREI 11 EFTA01146530 doi:10.1038/nature 11467 RESEARCH SUPPLEMENTARY INFORMATION Table S7. Time pressure and delay conditions versus baseline condition from Study I. (I) (2) (3) Time delay condition -6.351" -5.973** -6.456*** (2.511) (2.512) (2.434) Time pressure condition 4.930* 4.776* 4.471* (2.824) (2.759) (2.692) US residency (0=N, 1=Y) 4.981* 4.441** (2.610) (2.180) Age 0.284** 0.397*** (0.137) (0.106) Gender (0=M, 1=F) 0.769 0.572 (2.155) (1.767) Failed comprehension -3.294 -0.670 (2.343) (1.947) Disobeyed time pressure constraint -12.81*** (2.615) Disobeyed time delay constraint 5.920 (3.692) Education dummies No Yes Yes Constant 29.14*** 4.040 2.116 (2.027) (8.221) (6.402) Observations 655 655 892 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 6. Study 7: Time pressure / lime delay experiment with belief elicitation in the physical laboratory Methods Study 7 was conducted in the Harvard Decision Sciences Laboratory. Subjects were undergraduate and graduate students under 35 years old recruited from schools around the Boston metro area. Subjects received a $5 show up fee and then interacted anonymously via computers in the lab. The computer interface was identical to that used by subjects recruited on AMT in Study 6, with the following exceptions: Firstly, the stakes were 10-fold higher: each subject was given a $4 endowment, rather than the $0.40 endowment used in Study 6. Secondly, we assessed subjects' expectations about the contribution behavior of others in their group's 9. After making a decision about how much to contribute, subjects were taken to a screen in which they were asked to predict the average amount contributed by the three other members of their group. To incentivize this prediction, subjects were informed when reaching the prediction WWW.NATURE.COM/NATURE 114 EFTA01146531 doi:10.1038/nature 11467 RESEARCH SUPPLEMENTARY INFORMATION screen that they could earn up to an additional $2 depending on the accuracy of their prediction. Specifically, for every 10 cents by which their prediction differed from the actual average, they would lose 5 cents from their additional $2 payment. Results We begin with descriptive statistics: Subjects that obe ed time constraint All subjects Time pressure N=55 Time delay N=98 Time pressure N= 02 Time delay N=109 Mean Std Mean Std Mean Std Mean Std Contribution 230.73 154.85 169.49 153.45 197.73 151.32 163.39 157.21 Decision time 8.07 1.56 26.93 15.06 11.29 4.84 24.94 15.44 LoglO(Decision time) 0.90 0.10 1.38 0.20 1.02 0.17 1.33 0.25 Age 20.95 2.18 21.33 2.67 21.20 2.52 21.46 2.74 Gender (0=M, 1=F) 0.71 0.46 0.65 0.48 0.67 047 0.63 0.48 Failed Comprehension 0.38 0.49 0.32 0.47 0.35 0.48 0.32 0.47 Predicted avg contribution of others group members 201.38 114.22 183.33 116.97 182.12 110.28 177.60 116.41 Disobeyed time constraint - - 0.46 0.50 0.10 0.30 First we compare the contribution levels in the time pressure condition and the time delay condition. To do so, we perform a set of Tobit regressions with robust standard errors, taking contribution amount as the dependent variable (Table S8). Regression 1 shows that contributions were significantly higher in the time pressure condition. Regression 2 shows that this continues to be true when controlling for age, gender and failing to correctly answer the comprehension questions (although the p-value on the time pressure condition falls to p=0.052). Regression 3 shows that this effect is robust to including subjects that disobeyed the time constraint. Regressions 4 and 5 show that this continues to be true even when controlling for subjects' expectations about the average contribution of the other group members (Regression 4 includes only subjects that obeyed the time constraint, while regression 5 includes all subjects). The robustness to controlling for expectations about others' behavior indicates that the time constraint manipulation is actually making subjects more prosocial, rather than just making them more optimistic about how others will behave (and thus more inclined to reciprocate based on `conditional cooperation' 18-25. To provide direct evidence that the time constraint manipulation is not altering expectations about the behavior of others, we now perform another set of Tobit regressions with robust standard errors, this time taking predicted average contribution of the other group members as the dependent variable (Table S9). Regression 1 shows no difference in predictions between the two conditions. Regression 2 shows that this continues to be true when controlling for age, gender and failing to correctly answering the comprehension questions. Regression 3 shows that WWW.NATURLCOMMAIIIRE I IS EFTA01146532 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION this is robust to including subjects that disobeyed the time constraint. We also find no difference across conditions in predicted average contribution using a Rank-sum test (p ).360). Finally, we examine how subjects' contribution compares to their expectation of others. We find that the subjects under time pressure contribute significantly more than they expect others to contribute (Sign-rank, p=0.024), whereas subjects forced to reflect contribute slightly less than they expect others to contribute, although the difference is not statistically significant (Sign-rank, p=0.187). These results suggest that subjects responding intuitively are not just conforming to what they understand to be the norm, but rather are systematically deviating from the perceived norm and contributing more. Table S8. Contribution level in time pressure condition versus time delay condition, run in the hysical laboratory. (1) (2) (3) (4) (5) Time pressure condition 99.92** 94.36* 99.47** 71.05** 74.16** (49.44) (48.58) (45.81) (33.45) (33.22) Age 4.178 -2.275 5.301 3.236 (7.920) (6.272) (4.860) (4.349) Gender (0=M, 1=F) 5.766 43.95 53.25 63.42** (52.92) (41.77) (36.41) (31.55) Failed comprehension 126.9*** 79.80** 46.48 11.31 (48.43) (39.17) (32.05) (28.15) Disobeyed time constraint -I16.4** -53.64 (50.37) (38.74) Predicted avg contribution of others 1.655*** 1.496*** (0.167) (0.144) Constant 154.8*** 20.70 145.91 -307.4*** -230.6** (28.88) (179.0) (141.6) (117.2) (104.4) Observations 153 153 211 153 211 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<a 1 WWW.NATURe.COWNATURE 116 EFTA01146533 doi:10.1038/nature 11467 RESEARCH SUPPLEMENTARY INFORMATION Table S9. Predicted average contribution of other 3 group members in time pressure condition versus time delay condition, run in the wical laboratory. (1) (2) (3) Time pressure condition 23.89 22.16 23.86 (22.81) (22.23) (19.69) Age -0.114 -4.054 (4.149) (3.328) Gender (0=M, 1=F) -34.10 -19.31 (23.46) (17.83) Failed comprehension 53.98** 47A7** (24.45) (19.45) Disobe ed time constraint ..54.44*** 20.55) Hioni::::11: HHH10),", "," HHILH,H11 Ht6SHHH",""," (13.64) (90.52) (74.23) Observations 153 153 211 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 7. Behavior on AMT versus the physical laboratory (Study 6 vs Study 7) In combination, Study 6 and Study 7 allow us to compare behavior in an identical experiment between AMT and the physical lab with 10-fold higher stakes. To make contribution amounts directly comparable, we take the fraction of maximum possible contribution as our dependent variable (since contributions in Study 6 range from 0 to 40 cents, while contributions in Study 7 range from 0 to 400 cents). For the most basic measure, we collapse across time constraint conditions. We find that subjects in the lab contribute significantly less than those on AMT (47.9% of the endowment in the lab vs 58.9% on AMT gives a difference of 11.0%, Wilcoxon Rank-Sum p=0.001; the difference is extremely similar when including subjects that did not obey the time constraint: differences of 11.2%, p=0.0001). The magnitude of the difference is not trivial, but also is not exceptionally large. The lower level of cooperation we find among students in the lab is consistent with the results of a recent meta-analysis of the Dictator Gamen, in which students were found to be significantly less altruistic than non-students. More important than the absolute level of contribution, however, is the size of the effect of the time constraint manipulation. We see an almost identical difference between the time pressure and time delay conditions when comparing AMT and the lab (AMT: time pressure = 67.4%, time delay = 52.2%, difference = 15.2%; Lab: time pressure = 57.7%, time delay =42.3%, difference = 15.3%). To demonstrate that the effect of the time constraint does not vary significantly between AMT and the lab, we perform a set of Tobit regressions with robust standard errors (Table S10). Regression 2 finds no significant interaction between the time pressure condition dummy and a dummy for being run in the lab, and regression 4 shows that this remains true when controlling for age, gender, US residency, failing to correctly answering the WWW.NATURF-COWNATURIII7 EFTA01146534 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION comprehension questions and education level. For completeness, regressions without the interaction term are also included (regressions 1 and 3). Table S10. Contribution level (as a fraction of the total endowment) in the time pressure condition versus time delay condition, run on AMT (Study 6) and in the physical laboratory (Study 7). (I) (2) (3) (4) Lab (0=AMT, 1=Physical) M.192*** M.248*** M.177** M.232** (0.0637) (0.0827) (0.0783) (0.0938) Time pressure condition 0.269*** 0.264*** 0.278*** 0.275*** (0.0555) (0.0550) (0.0632) (0.0627) Age 0.00309 0.00312 (0) (0) Gender (0=M, 1=F) 0.0424 0.0424 (0.0579) (0.0579) US Residency 0.171** 0.169** (0.0675) (0.0675) Lab X Time pressure condition -0.0398 -0.0415 (0.133) (0.131) Education dummies No Yes No Yes Constant 0.572*** 0.294** 0.568*** -0.068 (0.0373) (0.121) (0.0387) (0.201) Observations 596 596 596 596 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 8. Study 8: Conceptual priming experiment on AMT Methods The experimental design for the conceptual priming experiment was identical to the baseline correlational decision time experiment (Study 1), except that an additional screen was added to the beginning of the experiment. To induce mindsets favoring more intuitive or more reflective decision-making, we employed an induction method introduced in a recent paper from our group22. In the previous study, we demonstrated the power of these specific primes to promote intuitive versus reflective thinking in the domain of religious belief, and our findings about intuition versus reflection were validated in a subsequent study from another group using a different methods . In the current study, we use the same priming procedure as we did in ref 22, and examine the effect of the primes on cooperation. A more intuitive or reflective cognitive style was induced as follows. Before the screen with the PGG instructions, subjects completed a screen in which they were asked to write a paragraph WWW.NATURe.COM/NATURE I III EFTA01146535 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION recalling an episode from their life. As per the procedure previously established in ref 22, subjects were instructed to write 8-10 sentences about one of four particular types of episodes (based on the treatment to which they were randomly assigned, see below), and only subjects that wrote at least 8 sentences were included in the analysis. We employed a 2 x 2 between- subjects design in which subjects were randomly assigned to write about a situation in which they adopted one of two cognitive approaches (intuitive vs. reflective) and where that approach lead to an outcome that was either negative or positive. The instructions for each of the resulting 4 conditions are listed below: Intuition-bad: Please write a paragraph (approximately 8-10 sentences) describing a time your intuition/first instinct led you in the wrong direction and resulted in a bad outcome. Reflection-bad: Please write a paragraph (approximately 8-10 sentences) describing a time carefully reasoning through a situation led you in the wrong direction and resulted in a bad outcome. Intuition-good: Please write a paragraph (approximately 8-10 sentences) describing a time your intuition/first instinct led you in the right direction and resulted in a good outcome. Reflection-good: Please write a paragraph (approximately 8-10 sentences) describing a time carefully reasoning through a situation led you in the right direction and resulted in a good outcome. The intuition-good and reflection-bad conditions were designed to increase the role of intuition relative to reflection. The intuition-good condition aimed to make subjects more inclined to follow their intuitive first response (and therefore less likely to reflect and carefully consider their decision). The reflection-bad condition aimed to make subjects less inclined to stop and reflect on whether their first response was well suited to the current situation (and therefore more likely to actually follow that intuitive first response). Conversely, the intuition-bad and reflection-good conditions were designed to increase the role of reflection relative to intuition. The intuition-bad condition aimed to make subjects more wary of their intuitive first response (and therefore more likely to reflect and question the suitability of that response). The reflection-good condition aimed to make subjects more inclined to carefully reason through their decision (and therefore less likely to automatically follow their intuitive first response). Critically, we make salient the general practice of trusting ones intuitions (or not), whatever those intuitions may be, rather than invoking experiences specifically related to cooperation. Additionally, we counterbalance valence, with both positive and negative outcomes in each of our two conditions. We note that decision times were not recorded in Study 8 due to a technical error, but that the effect of the primes on decision time is investigated in Study 9. WWW.NATURe.COWNATURE 119 EFTA01146536 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION Results We begin with descriptive statistics: Contribution Age Gender (O=M, 1=F) US Residency Failed Comprehension Paragraph length Intuition-Bad N=99 Reflect on-Bad N=77 Mean Std Mean Std 22.14 16.93 28.42 14.74 31.35 11.66 33.10 11.17 0.55 0.50 0.61 0.49 0.59 0.50 0.69 0.47 0.55 0.50 0.44 0.50 618 311 716 266 Reflection-Good N=69 Intuition-Good N=98 Mean Std Mean Std 20.41 15.54 23.47 15.99 31.43 10.39 30.96 11.07 0.64 0.48 0.62 0.49 0.70 0.46 0.64 0.48 0.42 0.50 0.51 0.50 670 215 631 245 The goal of Study 8 was to assess whether inducing a more intuitive mindset led to higher contribution compared to inducing a more reflective mindset. To do so, we perform two complementary analyses. Main effect of promoting intuition versus promoting reflection The first analysis uses a set of Tobit regressions with robust standard errors (Table S11). We begin by asking whether promoting intuition relative to reflection results in a different contribution level than promoting reflection relative to intuition. Regression I finds that the contribution level collapsing across the two conditions designed to promote intuition over reflection (intuition-good and reflection-bad) was significantly higher than when collapsing across to the two conditions designed to promoted reflection over intuition (reflection-good and intuition-bad). Regression 2 shows that this continues to be true when including a term for the valence of the outcome, controlling for variance explained by comparing the good outcome conditions (intuition-good and reflection-good) with the bad outcome conditions (intuition-bad and reflection-bad). Regression 3 shows that this again continues to be true when also controlling for US residency, age, gender, failing to correctly answer the comprehension questions, number of characters in the priming paragraph, and education level. We note that regressions 2 and 3 find a negative effect of positive outcome valence on cooperation (p).047 without controls in regression 2, p=0.074 with controls in regression 3). This result is consistent with a previous study finding that inducing positive mood resulted in less giving in a Dictator Game compared to inducing a negative mood24, although results from other studies on the role of mood in cooperation are mixed 5-27. The effect of mood on behavior in economic games merits further study. In regressions 4 and 5, we ask whether the effect of promoting intuition versus reflection differs based on the outcome valence. Either without controls (regression 4) or with controls (regression 5), we find no significant interaction between the promote intuition dummy and the outcome WWW.NATURe.COWNATURE 120 EFTA01146537 doi:10.1038/nature 11467 EMUPPLEMENTARY INFORMATION valence dummy. This lack of significant interaction term indicates that the difference between contributions in the intuition-good condition versus the reflection-good condition is not significantly different from the difference between contributions in the reflection-bad condition versus the intuition-bad condition. Put differently, the lack of significant interaction indicates that collapsing across the intuition-good and reflection-bad conditions, as well as across the reflection-good and intuition-bad conditions, is appropriate. Thus when we present the results of Study 8 in the main text, we do in this collapsed manner. Table SI I. Contribution level in conceptual primi rg experiment across pr ming conditions. (1) (2) (3) (4) (5) Promote intuition (0=[Intuition-bad, reflection-good],1=[Intuit ion-good , reflection-bad]) 10.95*** 12.16*** 11.14*** I5.61** 12.63** (4.184) (4.195) (4.031) (6.159) (6.018) Outcome valence (11 [Intuition-bad, reflection-bad],1=[Intuition-good, reflection-good]) -8.176** -7.262* -4.717 -5.781 (4.124) (4.059) (5.800) (5.721) US Residency (0=N, I=Y) 1333*** 13.65*** (4.942) (4.947) Age 0.356* 0.353* (0.194) (0.195) Gender (1-.M, I=F) 3.191 3.189 (4.205) (4.204) Failed comprehension -2.691 -2.587 (4.488) (4.500) Paragraph length -0.00131 -0.00158 (0.00912) (0.00908) Promote intuition X Outcome valence -6.906 -2.954 (8.244) (8.024) Education dummies No No Yes No Yes Constant 25.01*** 28.43*** 34.89** 26.96*** 34.51** (2.979) (3.591) (16.32) (4.076) (16.26) Observations 343 343 343 343 343 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 Interaction between cognitive style and outcome valence The second analysis demonstrates the same result in a different way, using the analytic approach of our earlier work in ref 22. Instead of looking for a main effect of promoting intuition versus WWW. NATURE.COMMAIIIPS1 21 EFTA01146538 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION reflection, we now ask whether there is a significant interaction between cognitive approach (intuition versus reflection) and outcome valence (bad versus good) using ANOVA. Specifically, contribution levels were subjected to a two-way ANOVA having two levels of reasoning style (intuitive/reflective) and two levels of outcome valence (bad/good). Together with no significant main effect of reasoning style (F(1,339)=0.85, p=0.357) and a marginally significant main effect of outcome valence (F(1,339)=3.69, p=0.056), we find a significant interaction between reasoning style and outcome valence (F(1,339)=7.21, p=0.008). This significant crossover interaction shows that participants who wrote about an experience that vindicated intuition (intuition-good or reflection-bad) contributed more to the public good compared with participants who wrote about an experience that vindicated reflection (intuition-bad or reflection- good). Thus we demonstrate in two different ways that in Study 8, priming to promote intuition increases contributions in the PGG relative to priming to promote reflection. 9. Study 9: Conceptual priming experiment with experience measure and decision times on AMT Methods Study 9 aimed to use the conceptual priming framework from Study 8 to examine the effect of previous experience with the experimental decision task on cooperative intuitions. Based on the theoretical framework presented in main text, where cooperative intuitions are developed in daily life because cooperation is advantageous and then these intuitions spill over into the laboratory, we predicted that the difference in contributions when promoting intuition versus promoting reflection should be smaller in experienced subjects. Study 9 also aimed to provide a manipulation check on the conceptual primes' ability to manipulate reaction times: based on Studies 1-7, we would expect promoting intuition not only to increase contributions relative to promoting reflection, but also to reduce decision times. To investigate these two issues, Study 9 used the design of the `intuition-good' and 'reflection- good' conditions from Study 8, with the following modifications. (i) In the post-experimental questionnaire subjects were asked "To what extent have you participated in studies like this one before? (i.e. where you choose how much to keep for yourself versus contributing to benefit others)". Subjects who chose the response "Never" were classified as naive. And (ii), decision times were recorded, as well as time spent reading the instructions. Results We begin with descriptive statistics: WWW .NATURE.COWNATURe 122 EFTA01146539 doi:10.1038/nature11467 En -SUPPLEMENTARY INFORMATION Contribution Decision time LoglO(Decision time) Instruction reading time LoglO(Instructions time) Age Gender (O=M, l=F) US Residency Failed Comprehension Paragraph length Naive subjects Reflection- Good N=38 Intuition- Good N=49 Mean Std Mean Std 19.79 16.76 29.92 15.29 15.16 13.70 11.69 737 1.07 0.30 1.00 0.24 69.45 35.85 87.06 93.57 1.78 0.24 1.86 0.23 29.08 9.67 28.73 9.68 0.42 0.50 0.43 0.50 0.82 0.39 0.65 0.48 0.39 0.50 0.37 0.49 722 222 645 233 Experienced subjects Reflection- Good N=94 Intuition- Good N=75 Mean Std Mean Std 24.21 16.11 24.00 16.46 13.10 12.66 13.88 23.00 0.99 0.31 0.99 0.30 67.76 63.26 67.47 36.88 1.73 0.28 1.77 0.24 30.33 11.09 33.29 12.49 0.55 0.50 0.53 0.50 0.84 0.37 0.81 0.39 0.26 0.44 0.28 0.45 699 248 694 218 The first goal for Study 9 was to test whether the prime condition had a greater effect among naive subjects compared to experienced subjects. To this end we use a set of Tobit regressions with robust standard errors (Table S12). We begin by analyzing all subjects together and examining the interaction between the prime condition (promote intuition versus promote reflection) and the subject's previous experience with the experimental task (naive versus experienced). As predicted, regression 1 shows a significant positive interaction between prime condition and naivety with respect to the experimental design, and regression 2 shows that this interaction remains significant when including controls for US residency, age, gender, failing to correctly answer the comprehension questions, number of characters in the priming paragraph and education level. Based on this significant interaction, we therefore analyze naive and experienced subjects separately. Regression 3 shows that among naive subjects, there is a significant positive effect of promoting intuition relative to promoting reflection. Regression 4 shows that this effect is robust to controls for US residency, age, gender, failing to correctly answer the comprehension questions, number of characters in the priming paragraph, and education level. Conversely, regressions 5 and 6 find no significant difference between priming conditions among experienced subjects, either without or with demographic controls. This finding is also consistent with the analyses in Studies 2 through 5, where the relationship between decision time and cooperation that is present at the beginning of the session becomes reduced or eliminated in later rounds. The second goal of Study 9 was to examine the effect of the prime on decision times. To do so, we perform a set of linear regressions with robust standard errors, taking loglO(Decision time) as the dependent variable and examining the data from the naive subjects (Table S13). Regression 1 finds a relationship which is non-significant but trending in the direction we expect based on Studies 1-7 (promoting intuition leading to shorter decision times). Regression 2 shows that this relationship becomes significant when including controls for US residency, age, gender, failing to correctly answer the comprehension questions, number of characters in the priming paragraph, time spend reading the instructions and education level. As we will show in Study 10 below, WWW.NATURE-COMMATURE I 23 EFTA01146540 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION time spent reading the instructions is positively correlated with decision time, but does not significantly predict contribution amount. Thus we include time spent reading the instructions as a control for the subject's general level of speed. Further support for the idea that time spent reading the instructions is a stable individual difference measure comes from the lack of relationship between prime condition and time spent reading the instructions demonstrated in Table S14. To further link the conceptual priming experiments to the experiments involving decision times, we now provide evidence that the prime condition in Study 9 affects contribution levels among naïve subjects specifically by manipulating decision times. Table S13 showed that promoting intuition resulted in faster decision times compared to promoting reflection. We now show in Table S15 that faster decision times are associated with higher contributions (as in Studies 1-5), and that the relationship between prime condition and contribution shown in Table S12 becomes non-significant when controlling for decision time. Thus it seems that priming intuition causes subjects to respond more quickly, and this quicker response leads to higher contribution. Table S12. Contribution level in conceptual priming experiment, naïve vs experienced subjects. All subjects Naive subjects Experienced subjects (1) (2) (3) (4) (5) (6) Prime condition (t- Reflection- good, 1=Intuition-good) -1.380 -1.932 28.57*** 22.66** -1.351 -1.922 (6.849) (6.860) (10.37) (10.69) (6.756) (6.777) Naive -9.930 -7.414 (8.034) (7.885) Prime condition X Naive 29.08** 26.55** (12.08) (11.91) Age 0.191 -0.289 0.284 (0.274) (0.727) (0.295) Gender (t-.M, 1=F) 10.93* 10.26 9.978 (5.699) (10.63) (6.707) US Residency 3.218 -8.374 7.237 (6.516) (11.58) (7.919) Failed comprehension 3.574 1.827 2.559 (5.824) (10.31) (6.998) Paragraph length 0.00160 -0.0297 0.0132 (0.0124) (0.0225) (0.0141) Education dummies No Yes No Yes No Yes Constant 32.07*** 54.07** 22.27*** 256.8*** 31.88*** 35.60* (4.638) (23.39) (6.951) (48.59) (4.620) (20.38) Observations 256 256 87 87 169 169 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 WWW.NATURLCOM/NATURE 124 EFTA01146541 doi:10.1038/nature 11467 RESEARCH SUPPLEMENTARY INFORMATION Table S13. Log1O(Decision time) in conceptual priming experiment, naïve subjects. (I) (2) Prime condition (0=Reflection-good, 1=Intuition-good) -0.0737 -0.130** (0.0593) (0.0629) Age -0.00216 (0.00273) Gender (0=M, 1=F) 0.00302 (0.0600) US Residency 0.0219 (0.0795) Failed comprehension -0.0250 (0.0586) Paragraph length -0.000218* (0.000129) log 10(Time reading instructions) 0.301** (0.148) Education dummies No Yes Constant 1.071*** 0.548* (0.0480) (0.298) Observations 87 87 R-squared 0.019 0.144 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 Table S14. Log1O(Time reading instructions) in conceptual priming experiment, naïve subjects. (I) (2) Prime condition (0=Reflection-good, 1=Intuition-good) 0.0711 0.0797 (0.0513) (0.0521) Age 0.00241 (0.00253) Gender (0=M, 1=F) -0.0122 (0.0550) US Residency -0.108* (0.0631) Failed comprehension -0.0258 (0.0595) Paragraph length 0.000112 (0.000157) logIO(Decision time) 0.223** (0.0964) Education dummies No Yes Constant 1.784*** 1.515*** (0.0392) (0.163) Observations 87 87 R-squared 0.022 0.185 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 WWW.NATURE-COM/NATUREI 23 EFTA01146542 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION Table S15. Contribution level in conceptual priming experiment as a function of decision time, naïve subjects. (1) (2) (3) (4) (5) log 10(Decision lime) -51.94*** -57.14*** -50.44*** (15.56) (15.64) (15.67) Prime condition (I: Reflection- good, 1=Intuition-good) 28.57*** 22.66** 15.94 (10.37) (10.69) (10.40) Age -0.450 -0.289 -0.377 (0.705) (0.727) (0.722) Gender (0=M, 1=F) 10.47 10.26 9.681 (10.24) (10.63) (10.07) US Residency -12.99 -8.374 -9.305 (10.90) (11.58) (10.70) Failed comprehension -1.016 1.827 0.656 (9.728) (10.31) (9.876) Paragraph length -0.0459** -0.0297 -0.0389* (0.0208) (0.0225) (0.0215) Education No Yes No Yes Yes Constant 92.38*** 321.2*** 22.27*** 256.8*** 299.8*** (19.03) (50.57) (6.951) (48.59) (52.26) Observations 87 87 87 87 87 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 10. Study 10: Correlational experiment on AMT with moderators, individual differences in cognitive style, and additional controls lethods The first aim of Study 10 was to test whether the relationship between decision time and cooperation differs depending on previous life experiences. Based on the theoretical framework presented in main text, we predicted that the difference in contributions between faster and slower responders should be smaller in subjects whose life outside the lab largely involves interactions with non-cooperative others. This prediction is rooted in the idea that mechanisms for the evolution of cooperation such as repetition and reputation typically involve multiple equilibria: Such mechanisms can make cooperation advantageous as long as enough others are WWW.NATLIRLCOM/NATURE 126 EFTA01146543 doi:10.1038/nature 11467 RESEARCH SUPPLEMENTARY INFORMATION also using cooperative strategies, but not if a sufficiently large number of others are selfish (as seen, for example, in the Folk Theorem, where fully cooperative strategies can be equilibria, but non-cooperative strategies are also always equilibria28). The second aim of Study 10 was test whether individual differences in cognitive style are predictive of cooperation. Substantial variation in cooperation has been documented across individuals (for a review, see ref 29). Similarly, the baseline propensity to follow one's intuitions versus stopping and reflecting has been shown to vary across individuals3a3l. Our manipulation experiments in Studies 6 thru 9 demonstrate through random assignment that at least some of the effect of cognitive style on cooperation occurs via variation within a given individual. However, the extent to which the results of the correlations in Studies 1 thru 5 are driven by variation in cognitive style within subjects versus variation across subjects has not been addressed. Here we investigate this question by asking whether contribution levels correlate with measures of individual differences in cognitive style. The third and final aim of Study 10 was to explore whether the relationship between decision time and contribution shown in Study 1 is driven by risk attitudes or attention/engagement rather than intuitive reasoning. To address these aims, Study 10 used the same design as Study 1, with the following additions: (i) To assess whether subjects developed their intuitions in more or less cooperative daily environments, the post-experimental questionnaire asked "To what extent do you feel you can trust other people that you interact with in your daily life?" using a ten-point Liken scale from "1=Very Little" to "10=Very Much". Cooperativeness of daily life interaction partners was operationalized using trust because we believe the concept of trust would be more familiar to our subjects and map more clearly onto what we mean by cooperation than terms such as "cooperative" or "altruistic" which are often uses differently in daily speech compared to how they are used in the academic literature. (ii) To assess the relationship between cognitive style and contribution, subjects completed the 3-item version of the Cognitive Reflection Test30 and the 40-item version of the Rational Experiential Inventory32 after making their decision and answering the comprehension questions. (iii) To control for risk attitudes, subjects completed a general risk-taking measure that has been widely validated33 asking "Are you generally a person who is fully prepared to take risks or do you try to avoid taking risks?" using an I I-point Liken scale from "O=Unwilling to take risks" to "10= Fully prepared to take risks", as well as a 10-item social risk taking scale34 after completing the cognitive style measures. (iv) To control for attentional processing and engagement in the task, we recorded the time subjects spent reading the instructions as well as the time spent making the decision. If the decision time result was due to attention/engagement, then the same effect should be present when examining reading times rather than decision times. Furthermore, to discourage potentially unengaged subjects from participating, we asked subjects to transcribe a paragraph of neutral handwritten text (reading "Yellow car not blue over and above everything else I might have said") prior to beginning the study, a procedure which has been suggested as a method for excluding AMT WWW.NATURE.CONVNATURE 131 EFTA01146544 doi:10.1038/nature 11467 IntlPPLEMENTARY INFORMATION Results subjects who disregard task instructions in order to complete tasks as quickly as possible'. We begin with descriptive statistics: N=341 Mean Std Contribution 23.86 16.25 Decision time 13.78 17.94 LogIO(Decision time) 1.02 0.28 Time reading instructions 73.21 92.15 LogIO(Time reading instructions) 1.73 0.35 Age 30.69 10.26 Gender 0=M, 1= 0.40 0.49 US Residenc 0=N 1= 0.70 0.46 Failed Comprehension (0=N, 1=Y) 0.34 0.47 View of daily interaction partners (1=Very untrustworthy to 7=Very trustworthy) 6 1 7 198 General risk taking (0 to 10) 6.64 2.44 Social risk taking (1 to 5) 3.20 0.56 Co nitive reflection test (0 to 3) 1.40 1.20 Need for cognition (1 to 10) 7.59 1.22 Faith in intuition (1 to 10) 6.70 1.23 The first goal of Study 10 was to test whether the relationship between decision time and contribution was stronger among subjects who view their daily life interaction partners as cooperative. To this end we perform a median split on the view of daily life interaction partners measure, separating subjects into those who view their daily interaction partners as more versus less cooperative, and perform a set of Tobit regressions with robust standard errors (Table S16). We begin by analyzing all subjects together and examining the interaction between decision time and view of daily life interaction partners. As predicted, regression I shows a significant positive main effect of having more cooperative daily life interaction partners together with a significant negative interaction between decision time and having more cooperative daily life interaction partners. Together, this main effect and interaction indicate that those who perceive their daily interaction partners as cooperative contribute more when they respond quickly (i.e. decision time is small), but that this increase in contribution is erased with longer decision times; whereas decision times have little effect on subjects from an uncooperative world. Regression 2 shows that the main effect and interaction term remain significant when including controls for US residency, age, gender, failing to correctly answer the comprehension questions, and education level. Based on the significant interaction, we therefore analyze subjects with more versus less cooperative daily interaction partners separately. Regression 3 shows that among subjects with largely cooperative partners outside of the lab, there is a significant negative relationship between decision time and contribution. Regression 4 shows that this effect is robust to controls for US residency, age, gender, failing to correctly answer the comprehension questions, and education level. Conversely, regressions 5 and 6 find no significant relationship between WWW.NATURECOM/NATURE 12S EFTA01146545 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION decision time and cooperation among subjects who perceive themselves as having less cooperative interaction partners outside the lab, either without or with demographic controls. Critically, a further analysis finds no relationship between view of one's daily interaction partners and decision time (logistic regression with robust standard errors; without controls: coefMI.159, p=0.682; with demographic controls: coeff=0.196, p=0.621). This demonstrates that attitude towards daily life interaction partners is suitable for use as a moderator in this context. The relationship between decision time and contribution among subjects with a more versus less cooperative daily life interaction partners is shown in Figure S3 (in contrast to Figure 3b in the main text which uses a median split on decision time, here we show the relationship over the full range of decision times). The second purpose of Study 10 was to test whether individual difference measures of cognitive style predict cooperation. As shown using Tobit regression with robust standard errors in Table S17, we find no significant relationship between contribution and score on the Cognitive Reflection Test30 (Regressions 1 and 2), the Need for Cognition scale32 (Regressions 3 and 4), or the Faith in Intuition scale32 (Regression 5 and 6). We find the same results when considering only subjects with a more positive view of their daily interaction partners (for brevity, analysis not shown). Using linear regression with robust standard errors (Table S18), we also find no significant relationship between any of the three measures and decision time (with the exception of a marginally significant negative relationship between Faith in Intuition and decision time when not including demographic controls). Again all of these results are qualitatively unchanged when restricting to subjects with a more positive view of their daily interaction partners. The lack of relationship between these individual difference measures of cognitive style and cooperation has important implications. Together with our manipulation experiments (in which subjects are randomly assigned to more intuitive or reflective thinking styles), these findings suggest that our correlational results are largely driven by within-subject variation across decisions in intuitiveness versus reflectiveness, rather than being the result of comparing fundamentally intuitive people with fundamentally reflective people. The third and final goal of Study 10 was to test whether the negative relationship between decision time and contribution in Study 1 is explained by risk attitudes or attention/engagement in the task. To do so, we perform a set of Tobit regressions with robust standard errors (Table S19). Regression I replicates the negative relationship between decision time and contribution found in Study 1. Regression 2 shows that the effect continues to hold when controlling for our standard controls. Regression 3 shows that the effect continues to hold when also controlling for the general risk-taking measure33, the social risk-taking measure34 and time spent reading the instructions. Regression 3 also finds that none of these measures are themselves significantly correlated with contribution. This demonstrates that none of these effects explain the observed relationship between decision time and contribution. Again, these results are all robust to considering only subjects with a more positive view of their daily life interaction partners (regressions not shown for brevity). To provide further evidence for time spent reading the instructions as a proxy for attention and engagement, we note the strong positive correlation between decision time and time spent reading the instructions (linear regression with robust standard errors taking logl0[decision time] as the DV and logIO[time reading instructions] as the IV; coeff=0.184, p=0.001). WWW.NATURLCOM/NATURE 129 EFTA01146546 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION Table S16. Contribution versus decision time for subjects with a less versus more positive view of Teo le the interact with in dail (1) (2) (3) (4) (5) (6) All subjects Subjects with cooperative daily interaction partners Subjects with uncooperative daily interaction partners loglO(Decision time) -4.112 -8.473 -51.28*** -55.24*** -3.470 -6.321 (10.36) (11.11) (14.57) (14.74) (9.181) (9.746) Opinion of daily interaction partners (0=Uncooperative, 1=Cooperative) 45.75** 43.89** (18.25) (18.34) log I0(Decision time) X Opinion of daily interaction partners -40.06** -36.34** (16.10) (16.42) Age 0.302 0.671* 0.143 (0.228) (0.396) (0.280) Gender (C.M, 1=F) 11.05** 5.655 14.99*** (4.528) (7.952) (5.234) US Residency -11.85** -20.97** -5.008 (4.742) (8.906) (5.288) Failed comprehension 8.696* 4.994 10.90** (4.499) (8.280) (5.100) Education dummies No Yes No Yes No Yes Constant 32.59*** 20.40 87.99*** 31.47 30.91*** 35.87* (11.78) (20.39) (16.77) (39.69) (10.41) (19.01) Observations 338 338 170 170 168 168 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 WWW.NATURE.COWNATUR I 30 EFTA01146547 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION a Contribution 100% 80% 60% 40% 20% 0% b 100% 80% 60% 40% Contribution 20% 0% 1 4,"••• 37 1. Cooperative daily interaction partners 40 . 54 r 4. . 23 9 f ift • 5 1 0 2 0.6 1 1.4 1.8 2.2 Decision time (loglO[sec]) 3 Uncooperative daily interaction partners 40 42 44. J. 124 22 21 2 0 2 0.6 1 1.4 1.8 2.2 Decision time (loglO[sec]) Figure S3. LogIO(Decision time) versus contribution for subjects with a more positive (a) versus negative (b) view of their daily life interaction partners. Error bars indicate standard error of the mean. Dot size is proportional to number of observations, which are indicated next to each dot. The trend line is not indicated in panel b as the relationship between decision time and contribution is not significant. WWW.NATURLCOMMAIIIREIJI EFTA01146548 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION Table S17. Contribution versus measures of individual differences in cognitive style. (1) (2) (3) (4) (5) (6) Cognitive reflection test 0.811 2.013 (1.878) (1.981) Need for cognition 1.280 3.016 (1.902) (1.987) Faith in intuition -0.931 -0.999 (1.921) (2.002) Age 0.294 0.288 0.321 (0.231) (0.233) (0.232) Gender (0=M, 1=F) 11.12** 11.74** 11.22** 4.651 4.677 4.664 US Residenc -8.754* -9.752** -8.264* (4.700) (4.770) (4.855) Failed comprehension 10.53** 10.37** 9.037** 4.709 4.549 4.465 Education dummies No Yes No Yes No Yes Constant 29.42*** 11.10 20.86 -8.048 36.76*** 19.18 (3.311) (17.01) (14.27) (21.99) (13.01) (20.09) Observations 341 341 341 341 341 341 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 WWW.NATURE-COM/NATURE I 32 EFTA01146549 doi:10.1038/nature 11467 RESEARCH SUPPLEMENTARY INFORMATION Table S18. Log1O(Decision time) versus measures of individual differences in cognitive style. (1) (2) (3) (4) (5) (6) Cognitive reflection test 0.00820 0.00101 (0.0125) (0.0132) Need for cognition -0.00913 -0.00787 (0.0119) (0.0120) Faith in intuition M.0211* -0.0111 (0.0122) (0.0131) Age 0.00208 0.00211 0.00220 (0.00151) (0.00150) (0.00154) Gender (0=M, 1=F) -0.0129 -0.0148 -0.00936 (0.0322) (0.0326) (0.0331) US Residency M.122*** M.121*** M.116*** (0.0369) (0.0368) (0.0367) Failed comprehension -0.0416 -0.0455 -0.0425 (0.0330) (0.0322) (0.0317) Education dummies No Yes No Yes No Yes Constant 1.007*** 1.117*** 1.088*** 1.174*** 1.160*** 1.182*** (0.0224) (0.231) (0.0924) (0.241) (0.0863) (0.237) Observations 338 338 338 338 338 338 R-squared 0.001 0.050 0.002 0.051 0.008 0.052 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 WWW.NATURC.COM/NATUREI 31 EFTA01146550 doi:10.1038/nature 11467 RESEARCH SUPPLEMENTARY INFORMATION Table S19. Contribution versus logIO(Decision time) with additional controls. (1) (2) (3) Decision time lo 10 seconds) (8.361) (8.710) (8.854) US Residency (0=N, 1=Y) -11.95** -10.86** (4.802) (4.836) Age 0.363 0.323 (0.230) (0.233) Gender (0=M, 1=F) 10.84** 10.15** (4.583) (4.712) Failed Comprehension (0=N, 1=Y) 7.668* 9.498** (4.509) (4.762) Social risk-taking (1 to 5) 0.952 (4.094) General risk-taking (1 to I I) 0.0910 (1.075) Time reading instructions (log10 seconds) 9.920 (6.111) Education dummies No Yes Yes Constant 52.24*** 40.78** 23.09 (9.403) (19.71) (25.66) Observations 338 338 338 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 WWW.NATURe.COWNATURE I 34 EFTA01146551 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION 11. Robustness of relationship between intuition and cooperation Did faster deciders simply not understand the instructions? In our each of our analyses, we demonstrate that the effect of intuition is robust to controlling for correctly answering the comprehension questions regarding the payoff structure. Thus it seems that comprehension does not explain our effect. How does decision time relate to time spent understanding the decision setting? In Study 10, we demonstrate that the decision time effect is robust to controlling for time spent reading the instructions (and that time spent reading the instructions does not predict contribution; if anything, it is trending in the opposite direction with longer readers contributing slightly more). Furthermore, the time constraint manipulation experiments in Studies 6 and 7 provide direct evidence that the relationship between decision time and contribution is independent of time spent reading the instructions. In the time constraint experiment, subjects are only informed that they must make their decisions quickly (or slowly) after they have finished reading the instructions and proceeded to the next screen. Thus time spent reading the instructions cannot account for the effect of time pressure/delay on cooperation we observe. The fact that the two processes of (a) understanding the game setup and payoffs, versus (b) actually making a decision, are distinct in our design is an important feature of our experiments. Because our decision time metric applies only to the latter, this allows us to explore the cognitive mechanism underpinning the contribution decision without adding confounds related to game comprehension. This stands in contrast to a previous reaction time study where in each round, subjects chose between a different set of 4 monetary divisions35. In that design, the `decision time' measures how long subjects take to read and understand the different payoff options, as well as how long it takes them to reach a decision. Thus their finding that longer decision times were associated with more prosocial divisions is not necessarily in conflict with ours: their result can be explained by prosocial subjects taking longer to understand the game setup (because of an interest in the other's payoffs as well as their own), rather than taking longer to reach their decision (conditional on understanding the game). Consistent with this interpretation is the positive (although not statistically significant) trend we find between contribution and time spent reading the instructions in Study 10. This perspective also helps connect our results to previous work showing that selfishness is automatic in the context of conflicts of interest, where reflection is required to perceive that such conflicts exist36. Further exploration of this difference in cognitive processing between the task of understanding the nature of the situation versus actually making a decision is an important direction for future research. Is the relationship between intuition and cooperation unique to our online sample, the small stakes used on AMT, or any particular feature(s) of our online experimental environment? Are our findings restricted to American college undergraduates? WWW.NATLIRLCOM/NATUREI 33 EFTA01146552 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION We find significantly more cooperation when forcing subjects to respond quickly compared to taking time to think both on AMT with subjects from around the world (Study 6) and in the physical lab using Boston-area undergraduates playing for 10x larger stakes (Study 7). This demonstrates the robustness of our fundamental finding with respect to subject pool, online vs offline, and stake size. Is the relationship between decision time and cooperation we observe the result of adjusting away from a salient anchor? It has been suggested that in resource division tasks, subjects anchor on an equitable split, and then with further reflection adjust away from that anchor towards selfishness37. However, anchoring and adjustment do not explain the relationship between decision time and cooperation that we observe in our experiments: In the lab experiments using Prisoner's Dilemma games (Studies 2 thru 4), subjects make a binary choice (cooperate or defect) rather than a numerical decision of how much to contribute. This binary decision does not involve anchoring and adjustment, yet we continue to find the same negative relationship between decision time and cooperation in this binary choice. Furthermore, even in the public goods games where subjects choose a numerical contribution amount and anchoring and adjustment is possible, it is not a priori evident why full contribution would be a more natural anchor than zero contribution. Are the effects of intuitive versus reflective thinking we observe the result of comparing more intuitive versus more reflective people (Le. variation across individuals), or the result of variation within individuals? Our various manipulation studies (Studies 6 through 9) demonstrate that there is within- individual variation in cognitive style and resulting behavior. Subjects are randomly assigned to conditions, and thus do not systemically vary across conditions in their dispositions toward intuitive vs. reflective decision-making. Yet the experimental manipulation can push people to be more intuitive and therefore more cooperative, or more reflective and therefore more selfish. Furthermore, we find no evidence of between-individual differences in use of intuitive versus reflection predicting cooperation in Study 10. Together, these findings suggest that our results are largely driven by within-subject variation across decisions in intuitiveness vs. reflectiveness, rather than reflecting differences between people who are dispositionally intuitive vs. dispositionally reflective. 12. Implications for economic and evolutionary models Our findings have significant implications for the understanding of human prosocial behavior. Economic models have typically explained non-self-interested actions using social preferences. For example, people in our experiments might cooperate if, rather than being purely selfish, they have consistent preferences for equity3839, effnciency40 and/or reciprocity; 43. However, our manipulation studies indicate that people do not have a single, consistent set of preferences. Instead, our data indicate that intuition often promotes behavior consistent with a set of preferences that are more prosocial than those favored by reflection. As we show, experimental WWW.NATURe.COWNATURE 136 EFTA01146553 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION manipulations that have no effect on material outcomes can cause different preferences to be operative. Thus our results highlight the need for more cognitively complex models of prosocial behavior, along the lines of recent models for non-social decision-making4447. Furthermore, our results suggest a special role for intuition in promoting cooperation" (in contrast to ref 48, in which reflection is assumed to underlie prosociality). The present experiments also have important implications for the evolution of cooperation. In traditional evolutionary models, each agent has a specific strategy that determines her behavior, such as cooperate or defect in one-shot games, or always-defect or tit-for-tat (or some other strategy) in repeated games. Natural selection then operates on these strategies. Our results, however, suggest that people are not of a single mind, and are not committed to a single strategy. Instead, social behaviors are the product of an internal equilibrium between competing strategies, with some strategies favored by intuition and others by reflection. Expanding evolutionary models to include this cognitive conflict is an important direction for future research, and can help us understand why natural selection would favor cooperative intuitions. 13. Previous dual-process research using economic games Previous research exploring automatic versus controlled processes and social preferences in economic games has largely focused on rejecting unfair offers in the Ultimatum Game (UG). Several behavioral and neurobiological studies suggest that rejections in this game are driven by intuitive processes4654, while others conclude that reflective processes promote rejection5556 or that rejections are not preferentially associated with either intuition or reflection57. This variation in results when studying the UG may be due to the presence of both prosocial motivations (e.g. fairness) and anti-social motivations (e.g. jealously or spite) leading to the same behavior. Rejecting low offers is certainly "other-regarding", and arguably fair, but is not "cooperative" (unlike contributions to the public good, which are clearly cooperative). Nonetheless, it seems that the bulk of evidence supports a dominant role of intuition in motivating rejections. At first, this finding may seem to contradict our main conclusion that cooperation is intuitive in social dilemmas: how can prosocial behavior (i.e. PGG cooperation) and antisocial behavior (ie rejecting in the UG) both be intuitive? In light of our proposed mechanism, however, it is possible that these observations represent two sides of the same coin. If our intuitions reflect behaviors that are beneficial in daily life, then (i) cooperation should be intuitive, because cooperation is typically advantageous in the context of repetition, reputation and sanctions; and at the same time (ii) rejecting low offers should also be intuitive, as once again this behavior is advantageous in interactions that involve recigrocity: rejecting a low offer today can lead others to make higher offers to you in the future 8. A general implication of this finding is that cooperation need not be the intuitive response under all circumstances. For example, defection might be the automatic action in a PD if one's partner defected against oneself in the previous period (as once again, this tit-for-tat style behavior can be optimal in repeated games); or cooperation may not be intuitive when interacting with out-group members. Exploring these issues is an important direction for future research. WWW.NATURE.COWNATURe 137 EFTA01146554 doi:10.1038/nature 11467 RESEARCH SUPPLEMENTARY INFORMATION We also note that other recent studies are consistent with our results regarding PGG cooperation. Such studies reveal a negative relationship between offers in the UG and decision time52•56, a positive effect of cognitive load on donations in the Dictator Gamew (although see ref 6I which finds no effect of cognitive load in the Dictator Game), a marginal negative effect of decision time on choosing the efficient option in a series of binary-choice money division tasks62, and a negative effect of decision time on choosing full cooperation in the centipede game57. Furthermore, our results are not explained by differences in the time taken to understand the game's payoff structure35, or by the degree of adjustment away from an equitable anchor37, as elaborated above in SI Section 11. 14. Supplemental study: Experiment on AMT showing that detailed comprehension questions induce reflective thinking and reduce cooperation This supplemental study explored the effect of asking detailed comprehension questions prior to the contribution decision (unlike the other studies conducted for this paper, in which comprehension questions were asked after the contribution decision). We hypothesized that forcing subjects to perform a detailed payoff calculation prior to making their decision would shift them into a more reflective mindset. Based on the results of our other studies, we thus predicted that comprehension questions prior to the decision would reduce the average contribution, and increase the average decision time. To assess these predictions, we had subjects on AMT participate in the same PGG as in Study I, with the addition of a detailed payoff calculation question to the comprehension check section ("If you contributed 20 cents, and the other 3 group members contributed 10, 30 and 40 cents respectively, what bonus would you earn? [Remember that (i) you start with 40 cents and (ii) for every two cents contributed, all group members receive 1 cent]"). We then compared behavior in two experimental conditions, with the position of the comprehension questions in the experimental protocol varied between conditions. In the 'Before decision' condition, the comprehension questions were included at the end of the screen with the instructions; thus in this condition, subjects had to reason thru the payoff calculation before advancing to the screen on which that made their contribution decision. In the 'After decision' condition, the comprehension questions appeared on the screen following the contribution decision (as in Studies 1 and 6 thru 10). The goal of this study was to assess the effect of reasoning through a payoff calculation prior to making one's decision. Thus in our analysis we will restrict our attention to the subset of subjects that correctly answered the comprehension questions (N=72 in the 'Before decision' condition, and N=51 in the 'After decision' condition): it is unclear that subjects who answered incorrectly actually reflected on the questions. In line with our first prediction, contributions are significantly lower in the 'Before decision' condition (51.0%) compared to the 'After decision' condition (69.3%; Rank-sum, p=0.0142). WWW.NATURe.COWNATURE I38 EFTA01146555 doi:10.1038/nature 11467 RESEARCH SUPPLEMENTARY INFORMATION This relationship continues to hold (coeff=22.61, f:-0.020) in a Tobit regression with robust standard errors including controls for age, gender and US residence. In line with our second prediction, subjects take significantly longer to reach their decisions in the `Before decision' condition (log 10(Decision time)=1.30) compared to the `After decision' condition (logIO(Decision time)=1.01; Rank-sum, p<0.001). This relationship again continues to hold (coef1=-0.252, p=0.001) in a linear regression with robust standard errors including controls for age, gender and US residence. Furthermore, despite the greater mean in the `Before decision' condition, there is significantly larger variance in loglO(Decision time) in the `Before decision' condition (variance=0.092) compared to the `After decision' condition (variance=0.172; F-test for the homogeneity of variances, p=0.016). Thus we provide evidence that completing a detailed comprehension question prior to making one's decision shifts subjects into a more reflective mindset and leads to less cooperation. It therefore seems likely that asking subjects to complete numerous detailed payoff calculations prior to making their decision (rather than the one question we asked here) would lead to even lower contribution levels and greater reflectiveness (and even less variance in decision time), potentially reducing or eliminating any association between decision time and cooperation by forcing all subjects into a reflective mindset. This should be kept in mind when analyzing decision time data from other datasets. WWW.NATURe.COWNATURE 139 EFTA01146556 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION 15. Experimental instructions Study 1 Screen 1: Thank you for accepting this HIT. You have received $0.50 for participating. You also have the opportunity to receive additional money, which will be described in the next few pages. Screen 2: You have been randomly assigned to interact with 3 other people. All of you receive this same set of instructions. You cannot participate in this study more than once. Each person in your group is given 40 cents for this interaction (in addition to the 50 cents you received already for participating). You each decide how much of your 40 cents to keep for yourself, and how much (if any) to contribute to the group's common project (in increments of 2 units: 0, 2, 4, 6 etc). All money contributed to the common project is doubled, and then split evenly among the 4 group members. Thus, for every 2 cents contributed to the common project, each group member receives I cent. If everyone contributes all of their 40 cents, everyone's money will double: each of you will earn 80 cents. But if everyone else contributes their 40 cents, while you keep your 40 cents, you will earn 100 cents, while the others will earn only 60 cents. That is because for every 2 cents you contribute, you get only 1 cent back. Thus you personally lose money on contributing. The other people are REAL and will really make a decision — there is no deception in this study. Once you and the other people have chosen how much to contribute, the interaction is over. Neither you nor the other people receive any bonus other than what comes out of this interaction. Screen 3: Please use the slider to choose the amount of money you wish to contribute. Your contribution: 0 slider -40 Screen 4: You MUST answer these two questions correctly to receive your bonus! WWW.NATURE-COMMATUREI 40 EFTA01146557 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION 1. What level of contribution earns the highest payoff for the group as a whole? 2. What level of contribution earns the highest payoff for you personally? Studies 2-5 See the original papers experimental instructions. Study 6 Screen 1: Thank you for accepting this HIT. You have received $0.50 for participating. You also have the opportunity to receive additional money, which will be described in the next few pages. Screen 2: You have been randomly assigned to interact with 3 other people. All of you receive this same set of instructions. You cannot participate in this study more than once. Each person in your group is given 40 cents for this interaction (in addition to the 50 cents you received already for participating). You each decide how much of your 40 cents to keep for yourself, and how much (if any) to contribute to the group's common project (in increments of 2 units: 0, 2, 4, 6 etc). All money contributed to the common project is doubled, and then split evenly among the 4 group members. Thus, for every 2 cents contributed to the common project, each group member receives I cent. If everyone contributes all of their 40 cents, everyone's money will double: each of you will earn 80 cents. But if everyone else contributes their 40 cents, while you keep your 40 cents, you will earn 100 cents, while the others will earn only 60 cents. That is because for every 2 cents you contribute, you get only I cent back. Thus you personally lose money on contributing. The other people are REAL and will really make a decision — there is no deception in this study. Once you and the other people have chosen how much to contribute, the interaction is over. Neither you nor the other people receive any bonus other than what comes out of this interaction. WWW.NATURS.COWNATURE 141 EFTA01146558 doi:10.1038/nature 11467 RESEARCH SUPPLEMENTARY INFORMATION Screen 3: [Time pressure condition] Please make your decision as quickly as possible. You must make your decision in less than 10 seconds! [Forced delay condition] Please carefully consider you decision. You must wait and think for at least 10 seconds before making your decision. Please use the slider to choose the amount of money you wish to contribute. Your contribution: 0 slider -40 Screen 4: You MUST answer these two questions correctly to receive your bonus! I. What level of contribution earns the highest payoff for the group as a whole? 2. What level of contribution earns the highest payoff for you personally? Study 7 Screen 1: In this task, you will participate in a simple decision making study. You will receive a $5 show- up fee, and then earn additional money based on your decision and the decision of others. You will be paid in cash immediately following the experiment. Screen 2: You have been randomly assigned to interact with 3 of the other people in the room. All of you receive this same set of instructions. You cannot participate in this study more than once. Each person in your group is given $4 for this interaction. You each decide how much of your $4 to keep for yourself, and how much (if any) to contribute to the group's common project (in increments of 2 cents: 0, 2, 4, 6 etc). All money contributed to the common project is doubled, and then split evenly among the 4 group members. Thus, for every 2 cents contributed to the common project, each group member receives I cent. If everyone contributes all of their $4, everyone's money will double: each of you will earn $8 But if everyone else contributes their $4, while you keep your $4, you will earn $10, while the others will earn only $6. That is because for every 2 cents you contribute, you get only 1 cent back. Thus you personally lose money on contributing. WWW.NATURe.COM/NATUREI 42 EFTA01146559 doi:10.1038/nature 11467 SUPPLEMENTARY INFORMATION Once you and the other people have chosen how much to contribute, the interaction is over. None of you can effect each other's payoffs other than through the single decision in this interaction. Screen 3: [Time pressure condition] Please make your decision as quickly as possible. You must make your decision in less than 10 seconds! [Forced delay condition] Please carefully consider you decision. You must wait and think for at least 10 seconds before making your decision. Please use the slider to choose the amount of money you wish to contribute. Your contribution: 0 Screen 4: lider -400 In this stage, we would like you to predict the average contribution of the others in your group. You can earn up to an additional $2 depending on the accuracy of your prediction. For every 10 cents by which your prediction differs from the actual average, you lose 5 cents from your additional $2 payment. Thus you have an incentive to be as accurate as possible when making your prediction. How much do you think the other people in your group contributed on average (0 to 400 cents)? Average contribution of other group members: 0 slider -400 Screen 5: I. What level of contribution earns the highest payoff for the group as a whole? 2. What level of contribution earns the highest payoff for you personally? Study 8 (and Study 9, using only the Intuition-good and Reflection-good conditions) Screen 1: Thank you for accepting this HIT. You have received $0.50 for participating. You also have the opportunity to receive additional money, which will be described in the next few pages. WWW.NATURe.COh4/NATURE I41 EFTA01146560 doi:10.1038/nature 11467 RESEARCH SUPPLEMENTARY INFORMATION Screen 2: In this task, you will participate in a simple decision making study, and then answer a short survey.When you finish the survey, you will receive a completion code in order to get paid. Screen 3: [Intuition-good condition] Please write a paragraph (approximately 8-10 sentences) describing a time your intuition/first instinct led you in the right direction and resulted in a good outcome. [Intuition-bad condition] Please write a paragraph (approximately 8-10 sentences) describing a time your intuition/first instinct led you in the wrong direction and resulted in a bad outcome. [Reflection-good condition] Please write a paragraph (approximately 8-10 sentences) describing a time when carefully reasoning through a situation led you in the right direction and resulted in a good outcome. [Reflection -bad condition] Please write a paragraph (approximately 8-10 sentences) describing a time when carefully reasoning through a situation led you in the wrong direction and resulted in a bad outcome. --Large text box-- Please click 'Next' to begin the study. Screen 4: You have been randomly assigned to interact with 3 other people. All of you receive this same set of instructions. You cannot participate in this study more than once. Each person in your group is given 40 cents for this interaction (in addition to the 50 cents you received already for participating). You each decide how much of your 40 cents to keep for yourself, and how much (if any) to contribute to the group's common project (in increments of 2 units: 0, 2, 4, 6 etc). All money contributed to the common project is doubled, and then split evenly among the 4 group members. Thus, for every 2 cents contributed to the common project, each group member receives I cent. If everyone contributes all of their 40 cents, everyone's money will double: each of you will earn 80 cents. But if everyone else contributes their 40 cents, while you keep your 40 cents, you will earn 100 cents, while the others will earn only 60 cents. That is because for every 2 cents you contribute, you get only I cent back. Thus you personally lose money on contributing. WWW.NATURE.COWNATURE 144 EFTA01146561 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION The other people are REAL and will really make a decision — there is no deception in this study. Once you and the other people have chosen how much to contribute, the interaction is over. Neither you nor the other people receive any bonus other than what comes out of this interaction. Screen 5: Please use the slider to choose the amount of money you wish to contribute. Your contribution: 0 slider -40 Screen 6: You MUST answer these two questions correctly to receive your bonus! I. What level of contribution earns the highest payoff for the group as a whole? 2. What level of contribution earns the highest payoff for you personally? Study 10 Screen 1: Thank you for accepting this HIT. You have received $0.50 for participating. You also have the opportunity to receive additional money, which will be described in the next few pages. Screen 2: Please transcribe this hand-written text in the box below. You must correctly transcribe the text in order for your HIT to be accepted. --Large text box-- Please click 'Next' to begin the study. WWW.NATURe.COWNATURE 143 EFTA01146562 doi:10.1038/nature 11467 RESEARCH SUPPLEMENTARY INFORMATION Screen 3: You have been randomly assigned to interact with 3 other people. All of you receive this same set of instructions. You cannot participate in this study more than once. Each person in your group is given 40 cents for this interaction (in addition to the 50 cents you received already for participating). You each decide how much of your 40 cents to keep for yourself, and how much (if any) to contribute to the group's common project (in increments of 2 units: 0, 2, 4, 6 etc). All money contributed to the common project is doubled, and then split evenly among the 4 group members. Thus, for every 2 cents contributed to the common project, each group member receives 1 cent. If everyone contributes all of their 40 cents, everyone's money will double: each of you will earn 80 cents. But if everyone else contributes their 40 cents, while you keep your 40 cents, you will earn 100 cents, while the others will earn only 60 cents. That is because for every 2 cents you contribute, you get only I cent back. Thus you personally lose money on contributing. The other people are REAL and will really make a decision — there is no deception in this study. Once you and the other people have chosen how much to contribute, the interaction is over. Neither you nor the other people receive any bonus other than what comes out of this interaction. Screen 4: Please use the slider to choose the amount of money you wish to contribute. Your contribution: 0 slider -40 Screen 5: You MUST answer these two questions correctly to receive your bonus! 3. What level of contribution earns the highest payoff for the group as a whole? 4. What level of contribution earns the highest payoff for you personally? WWW.NATURE.CONI/NA111RE 146 EFTA01146563 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION References 1 Rand, D. G. The promise of Mechanical Turk: How online labor markets can help theorists run behavioral experiments. Journal of theoretical biology 299, 172-179 (2012). 2 Suri, S. & Watts, D. J. Cooperation and Contagion in Web-Based, Networked Public Goods Experiments. PLoS ONE 6, e16836 (2011). 3 Horton, J. J., Rand, D. G. & Zeckhauser, R. J. The online laboratory: conducting experiments in a real labor market. Experimental Economics 14, 399-425 (2011). 4 Buhrmester, M. D., Kwang, T. & Gosling, S. D. Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data? Perspectives on Psychological Science 6, 3-5 (2011). 5 Amir, 0., Rand, D. G. & Gal, Y. a. K. Economic Games on the Internet: The Effect of $1 Stakes. PLoS ONE 7, e31461 (2012). 6 Mason, W. & Suri, S. Conducting Behavioral Research on Amazon's Mechanical Turk. Behavioral Research Methods 44, 1-23 (2011). 7 Paolacci, G., Chandler, J. & Ipeirotis, P. G. Running Experiments on Amazon Mechanical Turk. Judgment and Decision Making 5, 411-419 (2010). 8 Andreoni, J. Why free ride?: Strategies and learning in public goods experiments. Journal of Public Economics 37, 291-304 (1988). 9 Andreoni, J. Cooperation in Public-Goods Experiments: Kindness or Confusion? The American Economic Review 85, 891-904 (1995). 10 Andreoni, J. Warm-Glow versus Cold-Prickle: The Effects of Positive and Negative Framing on Cooperation in Experiments. The Quarterly Journal of Economics 110, 1-21 (1995). 11 Keser, C. & Winden, F. v. in Discussion Paper, University of Karlsruhe (1996). 12 Weimann, J. Individual behaviour in a free riding experiment. Journal of Public Economics 54, 185-200 (1994). 13 Pfeiffer, T., Tran, L., Krumme, C. & Rand, D. G. The value of reputation. Journal of the Royal Society Interface (2012). 14 Dreber, A., Rand, D. G., Fudenberg, D. & Nowak, M. A. Winners don't punish. Nature 452, 348-351 (2008). 15 Fudenberg, D., Rand, D. G. & Dreber, A. Slow to Anger and Fast to Forgive: Cooperation in an Uncertain World. American Economic Review 102, 720-749 (2012). 16 Rand, D. G., Dreber, A., Ellingsen, T., Fudenberg, D. & Nowak, M. A. Positive Interactions Promote Public Cooperation. Science 325, 1272-1275 (2009). 17 Fischbacher, U. z-Tree: Zurich toolbox for ready-made economic experiments. Experimental Economics 10, 171-178 (2007). 18 Croson, R. T. A. Theories of commitment, altruism and reciprocity: evidence from linear public goods games. Economic Inquiry 45, 199-216 (2007). 19 Fischbacher, U. & Gachter, S. Social Preferences, Beliefs, and the Dynamics of Free Riding in Public Goods Experiments. The American Economic Review 100, 541-556 (2010). 20 Fischbacher, U., Gachter, S. & Fehr, E. Are people conditionally cooperative? Evidence from a public goods experiment. Economics Letters 71, 397-404 (2001). 21 Engel, C. Dictator Games: A Meta Study. Experimental Economics 14, 583-610 (2011). WWW.NATURLCOM/NATURE 147 EFTA01146564 doi:10.1038/nature 11467 RESEARCH SUPPLEMENTARY INFORMATION 22 Shenhav, A., Rand, D. G. & Greene, J. D. Divine intuition: Cognitive style influences belief in God. Journal of Experimental Psychology: General 141, 423-428 (2012). 23 Gervais, W. M. & Norenzayan, A. Analytic Thinking Promotes Religious Disbelief. Science 336, 493-496 (2012). 24 Tan, H. B. & Forgas, J. P. When happiness makes us selfish, but sadness makes us fair: Affective influences on interpersonal strategies in the dictator game. Journal of Experimental Social Psychology 46, 571-576 (2010). 25 Hertel, G., Neuhof, J., Theuer, T. & Kerr, N. L. Mood effects on cooperation in small groups: Does positive mood simply lead to more cooperation? Cognition & Emotion 14, 441472 (2000). 26 Hertel, G. & Fiedler, K. Affective and cognitive influences in social dilemma game. European Journal of Social Psychology 24, 131-145 (1994). 27 Forgas, J. P. On feeling good and getting your way: Mood effects on negotiator cognition and bargaining strategies. Journal of Personality and Social Psychology 74, 565-577 (1998). 28 Fudenberg, D. & Maskin, E. The Folk Theorem in Repeated Games with Discounting or with Incomplete Information. Econometrica 54, 533-554 (1986). 29 Balliet, D., Parks, C. & Joirernan, J. Social value orientation and cooperation in social dilemmas: A meta-analysis. Group Processes and Intergroup Relations 12, 533-547 (2009). 30 Frederick, S. Cognitive Reflection and Decision Making. The Journal of Economic Perspectives 19, 2542 (2005). 31 Stanovich, K. E. & West, R. F. Individual Differences in Rational Thought. Journal of Experimental Psychology: General 127, 161-188 (1998). 32 Pacini, R. & Epstein, S. The relation of rational and experiential information processing styles to personality, basic beliefs, and the ratio-bias phenomenon. Journal of Personality and Social Psychology 76, 972-987 (1999). 33 Dohmen, T. et al. Individual risk attitudes: measurement, determinants, and behavioral consequences. Journal of the European Economic Association 9, 522-550 (2011). 34 Weber, E. U., Blais, A.-R. & Betz, N. E. A domain-specific risk-attitude scale: measuring risk perceptions and risk behaviors. Journal of Behavioral Decision Making 15, 263-290 (2002). 35 Piovesan, M. & Wengstrom, E. Fast or fair? A study of response times. Economics Letters 105, 193-196 (2009). 36 Moore, D. & Loewenstein, G. Self-Interest, Automaticity, and the Psychology of Conflict of Interest. Social Justice Research 17, 189-202 (2004). 37 Roch, S. G., Lane, J. A. S., Samuelson, C. D., Allison, S. T. & Dent, J. L. Cognitive Load and the Equality Heuristic: A Two-Stage Model of Resource Overconsumption in Small Groups. Organizational Behavior and Human Decision Processes 83, 185-212 (2000). 38 Fehr, E. & Schmidt, K. A theory of fairness, competition and cooperation. Quarterly Journal of Economics 114, 817-868 (1999). 39 Bolton, G. E. & Ockenfels, A. ERC: A Theory of Equity, Reciprocity, and Competition. The American Economic Review 90, 166-193 (2000). 40 Charness, G. & Rabin, M. Understanding Social Preferences with Simple Tests. Quarterly Journal of Economics 117, 817-869 (2002). WWW.NATURE.CONVNATURe I' EFTA01146565 doi:10.1038/nature11467 RESEARCH SUPPLEMENTARY INFORMATION 41 Levine, D. K. Modeling Altruism and Spitefulness in Experiments. Review of Economic Dynamics 1, 593-622 (1998). 42 Dufwenberg, M. & Kirchsteiger, G. A theory of sequential reciprocity. Games and Economic Behavior 47, 268-298 (2004). 43 Falk, A. & Fischbacher, U. A theory of reciprocity. Games and Economic Behavior 54, 293-315 (2006). 44 Benhabib, J. & Bisin, A. Modeling internal commitment mechanisms and self-control: A neuroeconomics approach to consumption—saving decisions. Games and Economic Behavior 52, 460-492 (2005). 45 Fudenberg, D. & Levine, D. K. A Dual-Self Model of Impulse Control. The American Economic Review 96, 1449-1476 (2006). 46 Loewenstein, G. F. & O'Donoghue, T. Animal Spirits: Affective and Deliberative Processes in Economic Behavior. Available at SSRN: http://ssrn.comlabstract=539843 (2004). 47 McClure, S. M., Laibson, D. I., Loewenstein, G. & Cohen, J. D. Separate Neural Systems Value Immediate and Delayed Monetary Rewards. Science 306, 503-507 (2004). 48 Bowles, S. & Gintis, H. in The Economy as a Evolving Complex System 3 (ed Lawrence Blume and Steven N. Durlauf) 339-364 (2002). 49 Sanfey, A. G., Rilling, J. K., Aronson, J. A., Nystrom, L. E. & Cohen, J. D. The Neural Basis of Economic Decision-Making in the Ultimatum Game. Science 300, 1755-1758 (2003). 50 Sutter, M., Kocher, M. & Straub, S. Bargaining under time pressure in an experimental ultimatum game. Economics Letters 81, 341-347 (2003). 51 Gospic, K. et al. Limbic Justice—Amygdala Involvement in Immediate Rejection in the Ultimatum Game. PLoS Biol 9, e1001054 (2011). 52 Alos-Ferrer, C., Achtziger, A. & Wagner, A. Social Preferences and Self-Control. SSRN eLibrary (2010). 53 Koenigs, M. & Tranel, D. Irrational Economic Decision-Making after Ventromedial Prefrontal Damage: Evidence from the Ultimatum Game. The Journal of Neuroscience 27, 951-956 (2007). 54 Grimm, V. & Mengel, F. Let me sleep on it: Delay reduces rejection rates in ultimatum games. Economics Letters 111, 113-115 (2011). 55 Wout, M. v. t., Kahn, R. S., Sanfey, A. G. & Aleman, A. Repetitive transcranial magnetic stimulation over the right dorsolateral prefrontal cortex affects strategic decision-making. Neuroreport 16, 1849-1852 (2005). 56 Knoch, D., Pascual-Leone, A., Meyer, K., Treyer, V. & Fehr, E. Diminishing Reciprocal Fairness by Disrupting the Right Prefrontal Cortex. Science 314, 829-832 (2006). 57 Rubinstein, A. Instinctive and cognitive reasoning: a study of response times. The Economic Journal 117, 1243-1259 (2007). 58 Nowak, M. A., Page, K. M. & Sigmund, K. Fairness Versus Reason in the Ultimatum Game. Science 289, 1773-1775 (2000). 59 Cappelletti, D., Guth, W. & Ploner, M. Being of two minds: Ultimatum offers under cognitive constraints. Journal of Economic Psychology 32, 940-950 (2011). 60 Schulz, J., Fischbacher, U., Thoni, C. & Utikal, V. in Research Paper Series, Thurgau Institute of Economics and Department of Economics at the University of Konstanz (2011). WWW.NATURE.CONVNATURe FS9 EFTA01146566 doi:10.1038/nature11467 Eta_SUPPLEMENTARY INFORMATION 61 Hauge, K. E., Brekke, K. A., Johansson, L.-O., Johansson-Stenman, 0. & Svedsater, H. in University of Gothenburg Working Papers in Economics (2009). 62 Zaki, J. & Mitchell, J. P. Equitable decision making is associated with neural markers of intrinsic value. Proceedings of the National Academy of Sciences 108, 19761-19766 (2011). WWW.NATURE.CONVNATURe I50 EFTA01146567

Technical Artifacts (36)

View in Artifacts Browser

Email addresses, URLs, phone numbers, and other technical indicators extracted from this document.

Domainwww.naturc.com
Domainwww.nature.com
Domainwww.naturs.com
GPS0.00131 -0.00158
GPS0.00160 -0.0297
GPS0.00208 0.00211
GPS0.00309 0.00312
GPS0.00820 0.00101
GPS0.00913 -0.00787
GPS0.0129 -0.0148
GPS0.0297 -0.0389
GPS0.0398 -0.0415
GPS0.0416 -0.0455
GPS0.0424 0.0424
GPS0.0711 0.0797
Phone1243-1259
Phone1272-1275
Phone1291-1298
Phone1301-1303
Phone1449-1476
Phone1605-1609
Phone1755-1758
Phone1773-1775
Phone1849-1852
Phone531-3535
URLhttp://dxdoiorg/10.1098/rsif2012.0332
URLhttp://ssrn.comlabstract=539843
Wire RefReferences
Wire RefReflection
Wire Refreflected
Wire Refreflecting
Wire Refreflection
Wire Refreflection57
Wire Refreflective
Wire Refreflectiveness

Forum Discussions

This document was digitized, indexed, and cross-referenced with 1,400+ persons in the Epstein files. 100% free, ad-free, and independent.

Annotations powered by Hypothesis. Select any text on this page to annotate or highlight it.