From: "Nowak, Martin A."

From: "Nowak, Martin A." To: Joi ito CC: JE NY <jeevacation®gmail.com> Subject: Re: Researchers can't evaluate novel ideas Date: Wed, 16 May 2018 12:38:52 +0000 'We find that evaluators systematically give lower scores to research proposals that are closer to their own areas of expertise and to those that are highly novel? this is exactly my impression. it makes a lot of sense! has to do with "mediocrity" we could also analyze this with game dynamics. On May 16, 2018, at 7:19 AM, Joi ito a> wrote: https://pubsonline.informs.orgidoi/abs/10.1287/mnsc.2015.2285 Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance, Novelty, and Resource Allocation in Science Selecting among alternative projects is a core management task in all innovating organizations. In this paper, we focus on the evaluation of frontier scientific research projects. We argue that the "intellectual distance" between the knowledge embodied in research proposals and an evaluator's own expertise systematically relates to the evaluations given. To estimate relationships, we designed and executed a grant proposal process at a leading research university in which we randomized the assignment of evaluators and proposals to generate 2,130 evaluator—proposal pairs. We find that evaluators systematically give lower scores to research proposals that are closer to their own areas of expertise and to those that are highly novel. The patterns are consistent with biases associated with boundedly rational evaluation of new ideas. The patterns are inconsistent with intellectual distance simply contributing "noise" or being associated with private interests of evaluators. We discuss implications for policy, managerial intervention, and allocation of resources in the ongoing accumulation of scientific knowledge. EFTA01057162