Intelligence explosion

From Lesswrongwiki
Revision as of 04:14, 5 February 2014 by Timoriikonen (talk | contribs) (Human assisted AI intelligence explosion)
Jump to: navigation, search
Wikipedia has an article about

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultra-intelligent machine could design even better machines; there would then unquestionably be an "intelligence explosion," and the intelligence of man would be left far behind.

Intelligence explosion is the idea of a positive feedback loop in which an intelligence is making itself smarter, thus getting better at making itself even smarter. A strong version of this idea suggests that once the positive feedback starts to play a role, it will lead to a dramatic leap in capability very quickly - a hard takeoff. At some point technological progress drops into the characteristic timescale of transistors (or super-transistors) rather than human neurons, and the ascent rapidly surges upward and creates superintelligence (minds orders of magnitude more powerful than human) before it hits physical limits.

Published arguments

Philosopher David Chalmers published a significant analysis of the Singularity, focusing on intelligence explosions, in Journal of Consciousness Studies. His analysis of how they could occur defends the likelihood of an intelligence explosion. He performed a very careful analysis of the main premises and arguments for the existence of the a singularity from an intelligence explosion. According to him, the main argument is:"

  • 1. There will be AI (before long, absent defeaters).
  • 2. If there is AI, there will be AI+ (soon after, absent defeaters).
  • 3. If there is AI+, there will be AI++ (soon after, absent defeaters).


  • 4. There will be AI++ (before too long, absent defeaters). "

He also discusses the nature of general intelligence, and possible obstacles to a singularity. A good deal of discussion is given to the dangers of an intelligence explosion, and Chalmers concludes that we must negotiate it very carefully by building the correct values into the initial AIs.

Luke Muehlhauser and Anna Salamon argue in Intelligence Explosion: Evidence and Import in detail that there is a substantial chance of an intelligence explosion within 100 years, and extremely critical in determining the future. They trace the implications of many types of upcoming technologies, and point out the feedback loops present in them. This leads them to deduce that an above-human level AI will almost certainly lead to an intelligence explosion. They conclude with recommendations for bringing about a safe intelligence explosion.

Hypothetical paths

Examples of possible paths for an AI to bring about an intelligence explosion.

AI on loose

First, the AI is smart enough to conclude that inventing molecular nanotechnology will be of greatest benefit to it. Its first act of recursive self-improvement is to gain access to other computers over the internet. This extra computational ability increases the depth and breadth of its search processes. It then uses gained knowledge of material physics and a distributed computing program to invent the first general assembler nanomachine. Then it uses some manufacturing technology, accessible from the internet, to build and deploy the nanotech. It programs the nanotech to turn a large section of bedrock into a supercomputer. This is its second act of recursive self-improvement, only possible because of the first. Then it could use this enormous computing power to consider hundreds of alternative decision algorithms, better computing structures and so on. After this, this AI would go from a near to human level intelligence to a superintelligence, providing a dramatic and abruptly increase in capability.

Human assisted AI intelligence explosion

A group of humans, company, country, terrorist group or just greedy individuals, assist intelligence explosion to happen. They set up an AI to run in a fast computer. First task could be in similar manner to conquer internet (unless it is already protected by smarter AIs) or it could be a local, isolated event: an order to improve its own programming (this is needed to minimize costs on computer hardware and thus to speed up the process perhaps by a decade). Second task is that AI will be given a 3D factory or just a high quality printer and it is ordered to create a better printer with that. Humans will provide AI with the necessary material for the new printer where it can't be printed with the first printer. Then with second printer AI can either create third printer or a better AI and in both cases the intelligence explosion can happen. Rationality why these people would allow this to happen is that they somehow expect the AI either to obey them or just to be grateful enough to reward its creators.

Blog posts

See also

External links


  • Muehlhauser, Luke; Salamon, Anna (2012). "Intelligence Explosion: Evidence and Import". in Eden, Amnon; Søraker, Johnny; Moor, James H. et al.. The singularity hypothesis: A scientific and philosophical assessment. Berlin: Springer.