Difference between revisions of "Seed AI"

From Lesswrongwiki
Jump to: navigation, search
 
(7 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
{{wikilink}}
 
{{wikilink}}
  
'''Seed AI''' is a term coined by [[Eliezer Yudkowsky]] for an [[AGI]] that would act as the starting point for a recursively self-improving AGI. Initially this program may have a sub-human intelligence. The key for successful [[AI takeoff]] would lie in creating adequate starting conditions.
+
A '''Seed AI''' (a term coined by [[Eliezer Yudkowsky]]) is an [[Artificial General Intelligence]] (AGI) which improves itself by [[Recursive self-improvement|recursively rewriting]] its own source code without human intervention. Initially this program would likely have a minimal intelligence, but over the course of many iterations it would evolve to human-equivalent or even trans-human reasoning. The key for successful [[AI takeoff]] would lie in creating adequate starting conditions.
  
The capabilities of a Seed AI may be contrasted with those of a human. While humans can increase their intelligence by, for example, learning mathematics, they cannot ''increase their ability to learn''. That is, humans cannot currently produce drugs that make us learn faster, nor can we implant intelligence increasing chips into our brains. Therefore we are not currently recursively self-improving. This is because we were evolved; brains were evolved before deliberative thought, and evolution cannot refactor its method of creating intelligence afterwards.
+
== History ==
  
An AI on the other hand, is created by humans' deliberative intelligence. Therefore we can in theory program a simple but general AI which has access to all its own programming. While is it true that any sufficiently intelligent being could determine how to recursively self-improve, some architectures, such as neural networks or evolutionary algorithms, may have a much harder time doing so. Seed AI is distinguished by being built to self-modify from the start.
+
The notion of machine learning without human intervention has been around nearly as long as the computers themselves. In 1959, [http://en.wikipedia.org/wiki/Arthur_Samuel Arthur Samuel] stated that "Machine learning is the field of study that gives computers the ability to learn without being explicitly programmed."<ref name=Smoothing>
 +
{{cite journal
 +
|last1=Han
 +
|first1=Zhimeng
 +
|title=Smoothing in Probability Estimation Trees
 +
|publisher=The University of Waikato
 +
|url=http://researchcommons.waikato.ac.nz/bitstream/handle/10289/5701/thesis.pdf?sequence=3}}
 +
</ref> Since that time, computers have been able to learn by a variety of methods, including [http://en.wikipedia.org/wiki/Artificial_neural_network neural networks] and [http://en.wikipedia.org/wiki/Bayesian_inference Bayesian inference].
  
One critical consideration in Seed AI is that its goal system must remain stable under modifications. The architecture must be proven to faithfully preserve its utility function while becoming more intelligent. If the first iteration of the Seed AI has a [[Friendly AI|friendly]] goal, and is sufficiently able to make predictions, then it will remain safe indefinitely; if it predicted that modifying would change its goal, it would not want that according to its current goal, and it would not self-modify.
+
While these approaches have enabled machines to become better at various tasks<ref name=eurisko>
 +
{{cite journal
 +
|last1=Lenat
 +
|first1=Douglas
 +
|title=Eurisko. A program that learns news heuristics and domain concepts.
 +
|publisher=Artificial Intelligence
 +
|url=http://researchcommons.waikato.ac.nz/bitstream/handle/10289/5701/thesis.pdf?sequence=3}}
 +
</ref>
 +
<ref name=checkers>
 +
{{cite journal
 +
|last1=Chellapilla
 +
|first1=Kumar
 +
|last2=Fogel
 +
|first2=David
 +
|title=Evolving an Expert Checkers Playing Program without Using Human Expertise
 +
|publisher=Natural Selection, Inc.
 +
|url=http://www.cs.ru.ac.za/courses/Honours/ai/HybridSystems/P2.pdf}}
 +
</ref>, it has not enabled them to overcome the limitations of these techniques, nor has it given them the ability to understand their own programming and make improvements. Hence, they are not able to adapt to new situations without human assistance.
  
==External Links==
+
== Properties ==
*[http://intelligence.org/upload/LOGI/seedAI.html Seed AI] design description by Eliezer Yudkowsky.
 
  
==See Also==
+
A Seed AI has abilities that previous approaches lack:
 +
 
 +
*''Understanding its own source code''. It must understand the purpose, syntax and architecture of its own programming. This type of self-reflection enables the AGI to comprehend its utility and thus preserve it.
 +
 
 +
*''Rewriting its own source code''. The AGI must be able to overhaul the very code it uses to fulfill its utility. A critical consideration is that it must remain stable under modifications, preserving its original goals.
 +
 
 +
This combination of abilities would, in theory, allow an AGI to recursively improve itself by becoming ''smarter'' within its original purpose. A [[Gödel machine]] rigorously defines a specification for such an AGI.
 +
 
 +
== Development ==
 +
 
 +
Currently, there are no known Seed AIs in existence, but it is an active field of research. Several organizations continue to pursue this goal, such as the [http://intelligence.org Singularity Institute], [http://opencog.org/ OpenCog], and [http://adaptiveai.com/ Adaptive AI].
 +
 
 +
== See Also ==
 
*[[Gödel machine]]
 
*[[Gödel machine]]
 
*[[AI takeoff]]
 
*[[AI takeoff]]
 
*[[Recursive self-improvement]]
 
*[[Recursive self-improvement]]
 
*[[Intelligence explosion]]
 
*[[Intelligence explosion]]
 +
*[[AGI]]
 +
 +
== References ==
 +
{{Reflist}}
 +
<ol start="4">
 +
<li>Yudkowsky, Eliezer.[http://intelligence.org/upload/LOGI/seedAI.html Seed AI Levels of Organization in General Intelligence]. Singularity Institute.</li>
 +
<li>[http://intelligence.org/files/GISAI.html#para_seedAI_advantage General Intelligence and Seed AI]. Singularity Institute.</li>
 +
<li>Schmidhuber, Jürgen. [ftp://ftp.idsia.ch/pub/juergen/gm6.pdf Gödel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements]. IDSIA</li>
 +
</ol>
 +
[[Category:AI]]
 +
[[Category:AGI]]

Latest revision as of 01:42, 22 June 2017

Smallwikipedialogo.png
Wikipedia has an article about


A Seed AI (a term coined by Eliezer Yudkowsky) is an Artificial General Intelligence (AGI) which improves itself by recursively rewriting its own source code without human intervention. Initially this program would likely have a minimal intelligence, but over the course of many iterations it would evolve to human-equivalent or even trans-human reasoning. The key for successful AI takeoff would lie in creating adequate starting conditions.

History

The notion of machine learning without human intervention has been around nearly as long as the computers themselves. In 1959, Arthur Samuel stated that "Machine learning is the field of study that gives computers the ability to learn without being explicitly programmed."[1] Since that time, computers have been able to learn by a variety of methods, including neural networks and Bayesian inference.

While these approaches have enabled machines to become better at various tasks[2] [3], it has not enabled them to overcome the limitations of these techniques, nor has it given them the ability to understand their own programming and make improvements. Hence, they are not able to adapt to new situations without human assistance.

Properties

A Seed AI has abilities that previous approaches lack:

  • Understanding its own source code. It must understand the purpose, syntax and architecture of its own programming. This type of self-reflection enables the AGI to comprehend its utility and thus preserve it.
  • Rewriting its own source code. The AGI must be able to overhaul the very code it uses to fulfill its utility. A critical consideration is that it must remain stable under modifications, preserving its original goals.

This combination of abilities would, in theory, allow an AGI to recursively improve itself by becoming smarter within its original purpose. A Gödel machine rigorously defines a specification for such an AGI.

Development

Currently, there are no known Seed AIs in existence, but it is an active field of research. Several organizations continue to pursue this goal, such as the Singularity Institute, OpenCog, and Adaptive AI.

See Also

References

  1. Yudkowsky, Eliezer.Seed AI Levels of Organization in General Intelligence. Singularity Institute.
  2. General Intelligence and Seed AI. Singularity Institute.
  3. Schmidhuber, Jürgen. Gödel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements. IDSIA