AGI Ethics and Global Brain Scenarios Discussed in House Oversight Document

12.9 AGI Ethics As Related to Various Future Scenarios 237 pragmatic underlying ethical pedagogy and experiential grounding may prove to be even more dangerous than our messy human mental ecologies. If one assumes a hard takeoff AGI, then all this must be codified in the system at launch, as once a potentially Singularitarian AGI is launched there is no way to know what time period constitutes “before the singularity point.” This means developing theory of mind empathy and logical ethics in code prior to giving the system unfettered access to hardware and self- modification code. However, though nobody can predict if or when a Singularity will occur after unrestricted launch, only a truly irresponsible AGI development team would attempt to create an AGI without first experimenting with ethical training of the system in an intelligence- capped form, by means of ethical instruction via human-AGI interaction both pedagogically and experientially. 12.9.4 Global Brain Mindplex Scenarios Another class of scenarios — overlapping some of the previous ones — involves the emergence of a “Global Brain,” an emergent intelligence formed from global communication networks in- corporating humans and software programs in a larger body of self-organizing dynamics. The notion of the Global Brain is reviewed in [Hey07, Tur77] and its connection with advanced AI is discussed in detail in Goertzel’s book Creating Internet Intelligence [Goe01], where three possible phases of “Global Brain” development are articulated: e Phase 1: computer and communication technologies as enhancers of human interactions. This is what we have today: science and culture progress in ways that would not be possible if not for the “digital nervous system” we’re spreading across the planet. The network of idea and feeling sharing can become much richer and more productive than it is today, just through incremental development, without any Metasystem transition. e Phase 2: the intelligent Internet. At this point our computer and communication sys- tems, through some combination of self-organizing evolution and human engineering, have become a coherent mind on their own, or a set of coherent minds living in their own digital environment. e Phase 3: the full-on Singularity. A complete revision of the nature of intelligence, human and otherwise, via technological and intellectual advancement totally beyond the scope of our current comprehension. At this point our current psychological and cultural realities are no more relevant than the psyche of a goose is to modern society. The main concern of Creating Internet Intelligence is with e how to get from Phase 1 to Phase 2 - i.e. how to build an AGI system that will effect or encourage the transformation of the Internet into a coherent intelligent system e how to ensure that the Phase 2, Internet-savvy, global-brain-centric AGI systems will be oriented toward intelligence-improving self-modification (so they’ll propel themselves to Phase 3), and also toward generally positive goals (as opposed to, say, world domination and extermination of all other intelligent life forms besides themselves!) One possibly useful concept in this context is that of a mindplex: an intelligence that is composed largely of individual intelligences with their own self-models and global workspaces, HOUSE_OVERSIGHT_013153