Difference between revisions of "Recursive self-improvement"

From Lesswrongwiki
Jump to: navigation, search
Line 1: Line 1:
'''Recursive self-improvement''' refers to the property of making improvements on one's own ability of making self-improvements. It is an approach to [[Artificial General Intelligence]] that allows a system to make adjustments to its own functionality resulting in improved performance. The system could then feedback on itself with each cycle reaching ever higher levels of intelligence resulting in either a hard or soft [[AI takeoff]]. An agent can self-improve  and get a linear succession of improvements, however if it is able to improve its ability of making self-improvements, then each step will yield exponentially more improvements then the next one.
+
'''Recursive self-improvement''' refers to the property of making improvements on one's own ability of making self-improvements. It is an approach to [[Artificial General Intelligence]] that allows a system to make adjustments to its own functionality resulting in improved performance. The system could then feedback on itself with each cycle reaching ever higher levels of intelligence resulting in either a hard or soft [[AI takeoff]].
 +
 
 +
An agent can self-improve  and get a linear succession of improvements, however if it is able to improve its ability of making self-improvements, then each step will yield exponentially more improvements then the next one.
  
 
==Recursive self-improvement and [[AI takeoff]]==
 
==Recursive self-improvement and [[AI takeoff]]==
Recursively self-improving AI is considered to be the push behind the [[intelligence explosion]]. While any sufficiently intelligent AI will be able to improve itself, [[Seed AI|Seed AIs]] are specifically designed to use recursive self-improvement as their primary method of gaining intelligence. Architectures that had not been designed with this goal in mind, such as neural networks or large "hand-coded" projects like [[Cyc]], would have a harder time self-improving. [[Eliezer Yudkowsky]] argues that a recursively self-improvement AI seems likely to deliver a hard AI takeoff – a fast, abruptly, local increase in capability -, since the exponential increase in intelligence would yield an exponential return in benefits and resources that would feed even more returns in the next step and so on. In his view a soft takeoff scenario seems unlikely: it should either flatline or blow up.  You would need exactly the right law of diminishing returns to fly through the extremely narrow soft takeoff keyhole.[http://lesswrong.com/lw/we/recursive_selfimprovement/]. There are several points which seem to support the [[AI_takeoff#Hard_takeoff|hard takeoff scenario]]. Some of them are the fact that one improvement seems to lead the way to another, [[computing overhang|hardware overhang]] and the fact that sometimes- when navigating through problem space - one can find a succession of extremely easy to solve problems. These are all reasons for suddenly and abruptly increases in capability. On the other hand, [[Robin Hanson]] argues that there will be mostly  a slow and gradual accumulation of improvements, without a sharp change.
+
Recursively self-improving AI is considered to be the push behind the [[intelligence explosion]]. While any sufficiently intelligent AI will be able to improve itself, [[Seed AI|Seed AIs]] are specifically designed to use recursive self-improvement as their primary method of gaining intelligence. Architectures that had not been designed with this goal in mind, such as neural networks or large "hand-coded" projects like [[Cyc]], would have a harder time self-improving.
 +
 
 +
[[Eliezer Yudkowsky]] argues that a recursively self-improvement AI seems likely to deliver a hard AI takeoff – a fast, abruptly, local increase in capability - since the exponential increase in intelligence would yield an exponential return in benefits and resources that would feed even more returns in the next step, and so on. In his view a soft takeoff scenario seems unlikely: "it should either flatline or blow up.  You would need exactly the right law of diminishing returns to fly through the extremely narrow soft takeoff keyhole."[http://lesswrong.com/lw/we/recursive_selfimprovement/].
 +
 
 +
Yudkowsky argues that there are several points which seem to support the [[AI_takeoff#Hard_takeoff|hard takeoff scenario]]. Some of them are the fact that one improvement seems to lead the way to another, [[computing overhang|hardware overhang]] and the fact that sometimes- when navigating through problem space - one can find a succession of extremely easy to solve problems. These are all reasons for suddenly and abruptly increases in capability. On the other hand, [[Robin Hanson]] argues that there will be mostly  a slow and gradual accumulation of improvements, without a sharp change.
  
 
==Self-improvement in humans==
 
==Self-improvement in humans==
The human species has made an enormous amount of progress since evolving around fifty thousand years ago. This is because we can pass on knowledge and infrastructure from previous generations. This is a type of self-improvement, but it is not ''recursive''. If we never learned to modify our own brains, then we would eventually reach the point where making new discoveries required more knowledge than could be gained in a human lifetime. All human progress to date has been limited by the hardware we are born with, which is the same hardware Homo sapiens were born with fifty thousand years ago. ''True'' recursive self-improvement will come when we discover how to modify or augment our own brains in order to be more intelligent. This would lead us to more quickly being able to discover how to become even more intelligent. Assuming the rate is fast enough, this initiates a positive feedback loop.
+
The human species has made an enormous amount of progress since evolving around fifty thousand years ago. This is because we can pass on knowledge and infrastructure from previous generations. This is a type of self-improvement, but it is not ''recursive''. If we never learned to modify our own brains, then we would eventually reach the point where making new discoveries required more knowledge than could be gained in a human lifetime. All human progress to date has been limited by the hardware we are born with, which is the same hardware Homo sapiens were born with fifty thousand years ago.
 +
 
 +
"True" recursive self-improvement will come when we discover how to drastically modify or augment our own brains in order to be more intelligent. This would lead us to more quickly being able to discover how to become even more intelligent.
  
 
==Recursive self-improvement and [[Instrumental value]]==
 
==Recursive self-improvement and [[Instrumental value]]==
[[Nick Bostrom]] and [[Wikipedia:Steve Omohundro|Steve Omohundro]] have separately[http://www.nickbostrom.com/superintelligentwill.pdf] argued[http://selfawaresystems.files.wordpress.com/2008/01/nature_of_self_improving_ai.pdf] that despite the fact that values and intelligence are independent, any recursive self-improvement intelligence would likely possess a common set of instrumental values which are useful for achieving any kind of [[terminal value]]. For Omohundro those instrumental values are: Efficiency, Self-Preservation, Acquisition and Creativity; and for Bostrom: Self-preservation, Goal-content integrity, Cognitive enhancement, Technological perfection and Resource acquisition. They both argue these values should be used in trying to predict a superinteligence behavior; and that these values by themselves don't provide any safety indication, Omohundro says: "The best of these traits could usher in a new era of peace and prosperity; the worst are characteristic of human psychopaths and could bring widespread destruction".
+
{{Main|Basic AI drives}}
 +
 
 +
[[Nick Bostrom]] and [[Wikipedia:Steve Omohundro|Steve Omohundro]] have separately[http://www.nickbostrom.com/superintelligentwill.pdf] argued[http://selfawaresystems.files.wordpress.com/2008/01/nature_of_self_improving_ai.pdf] that despite the fact that values and intelligence are independent, any recursively self-improving intelligence would likely possess a common set of instrumental values which are useful for achieving any kind of [[terminal value|goal]]. As a system's intelligence continued modifying itself towards greater intelligence, it would be likely to adopt more of these behaviors.
  
 
==Blog posts==
 
==Blog posts==

Revision as of 18:46, 11 October 2012

Recursive self-improvement refers to the property of making improvements on one's own ability of making self-improvements. It is an approach to Artificial General Intelligence that allows a system to make adjustments to its own functionality resulting in improved performance. The system could then feedback on itself with each cycle reaching ever higher levels of intelligence resulting in either a hard or soft AI takeoff.

An agent can self-improve and get a linear succession of improvements, however if it is able to improve its ability of making self-improvements, then each step will yield exponentially more improvements then the next one.

Recursive self-improvement and AI takeoff

Recursively self-improving AI is considered to be the push behind the intelligence explosion. While any sufficiently intelligent AI will be able to improve itself, Seed AIs are specifically designed to use recursive self-improvement as their primary method of gaining intelligence. Architectures that had not been designed with this goal in mind, such as neural networks or large "hand-coded" projects like Cyc, would have a harder time self-improving.

Eliezer Yudkowsky argues that a recursively self-improvement AI seems likely to deliver a hard AI takeoff – a fast, abruptly, local increase in capability - since the exponential increase in intelligence would yield an exponential return in benefits and resources that would feed even more returns in the next step, and so on. In his view a soft takeoff scenario seems unlikely: "it should either flatline or blow up. You would need exactly the right law of diminishing returns to fly through the extremely narrow soft takeoff keyhole."[1].

Yudkowsky argues that there are several points which seem to support the hard takeoff scenario. Some of them are the fact that one improvement seems to lead the way to another, hardware overhang and the fact that sometimes- when navigating through problem space - one can find a succession of extremely easy to solve problems. These are all reasons for suddenly and abruptly increases in capability. On the other hand, Robin Hanson argues that there will be mostly a slow and gradual accumulation of improvements, without a sharp change.

Self-improvement in humans

The human species has made an enormous amount of progress since evolving around fifty thousand years ago. This is because we can pass on knowledge and infrastructure from previous generations. This is a type of self-improvement, but it is not recursive. If we never learned to modify our own brains, then we would eventually reach the point where making new discoveries required more knowledge than could be gained in a human lifetime. All human progress to date has been limited by the hardware we are born with, which is the same hardware Homo sapiens were born with fifty thousand years ago.

"True" recursive self-improvement will come when we discover how to drastically modify or augment our own brains in order to be more intelligent. This would lead us to more quickly being able to discover how to become even more intelligent.

Recursive self-improvement and Instrumental value

Main article: Basic AI drives

Nick Bostrom and Steve Omohundro have separately[2] argued[3] that despite the fact that values and intelligence are independent, any recursively self-improving intelligence would likely possess a common set of instrumental values which are useful for achieving any kind of goal. As a system's intelligence continued modifying itself towards greater intelligence, it would be likely to adopt more of these behaviors.

Blog posts

See also

External links