Difference between revisions of "Seed AI"

From Lesswrongwiki
Jump to: navigation, search
(See Also)
 
(11 intermediate revisions by 6 users not shown)
Line 1: Line 1:
 
{{wikilink}}
 
{{wikilink}}
'''Seed AI''' is a term coined by Eliezer Yudkowsky for a program that would act as the starting point for a recursively self-improving AGI. Initially this program would have a sub-human intelligence. The key for successful [[AI takeoff]] would lie in creating adequate starting conditions, this would not just mean a program capable of self-improving, but also doing so in a way that would produce [[Friendly AI]].
 
  
'''Seed AI''' differs from previously suggested methods of AI control, such as Asimov's 3 Laws of Robotics, in that it is assumed a suitably motivated SAI would be able to circumvent any core principles forced upon it. Instead, it would be free to harm a human, but would strongly hold the desire not to. This would allow for circumstances where some greater good may result by causing harm. However this raises issues of moral relativism.
+
A '''Seed AI''' (a term coined by [[Eliezer Yudkowsky]]) is an [[Artificial General Intelligence]] (AGI) which improves itself by [[Recursive self-improvement|recursively rewriting]] its own source code without human intervention. Initially this program would likely have a minimal intelligence, but over the course of many iterations it would evolve to human-equivalent or even trans-human reasoning. The key for successful [[AI takeoff]] would lie in creating adequate starting conditions.
  
==External Links==
+
== History ==
  
*[http://intelligence.org/upload/LOGI/seedAI.html Seed AI] design description by Eliezer Yudkowsky.
+
The notion of machine learning without human intervention has been around nearly as long as the computers themselves. In 1959, [http://en.wikipedia.org/wiki/Arthur_Samuel Arthur Samuel] stated that "Machine learning is the field of study that gives computers the ability to learn without being explicitly programmed."<ref name=Smoothing>
 +
{{cite journal
 +
|last1=Han
 +
|first1=Zhimeng
 +
|title=Smoothing in Probability Estimation Trees
 +
|publisher=The University of Waikato
 +
|url=http://researchcommons.waikato.ac.nz/bitstream/handle/10289/5701/thesis.pdf?sequence=3}}
 +
</ref> Since that time, computers have been able to learn by a variety of methods, including [http://en.wikipedia.org/wiki/Artificial_neural_network neural networks] and [http://en.wikipedia.org/wiki/Bayesian_inference Bayesian inference].
  
==See Also==
+
While these approaches have enabled machines to become better at various tasks<ref name=eurisko>
*[[Goedel Machine]]
+
{{cite journal
*[[Hard takeoff]]
+
|last1=Lenat
*[[Soft takeoff]]
+
|first1=Douglas
*[[Friendly AI]]
+
|title=Eurisko. A program that learns news heuristics and domain concepts.
*[[Unfriendly AI]]
+
|publisher=Artificial Intelligence
 +
|url=http://researchcommons.waikato.ac.nz/bitstream/handle/10289/5701/thesis.pdf?sequence=3}}
 +
</ref>
 +
<ref name=checkers>
 +
{{cite journal
 +
|last1=Chellapilla
 +
|first1=Kumar
 +
|last2=Fogel
 +
|first2=David
 +
|title=Evolving an Expert Checkers Playing Program without Using Human Expertise
 +
|publisher=Natural Selection, Inc.
 +
|url=http://www.cs.ru.ac.za/courses/Honours/ai/HybridSystems/P2.pdf}}
 +
</ref>, it has not enabled them to overcome the limitations of these techniques, nor has it given them the ability to understand their own programming and make improvements. Hence, they are not able to adapt to new situations without human assistance.
 +
 
 +
== Properties ==
 +
 
 +
A Seed AI has abilities that previous approaches lack:
 +
 
 +
*''Understanding its own source code''. It must understand the purpose, syntax and architecture of its own programming. This type of self-reflection enables the AGI to comprehend its utility and thus preserve it.
 +
 
 +
*''Rewriting its own source code''. The AGI must be able to overhaul the very code it uses to fulfill its utility. A critical consideration is that it must remain stable under modifications, preserving its original goals.
 +
 
 +
This combination of abilities would, in theory, allow an AGI to recursively improve itself by becoming ''smarter'' within its original purpose. A [[Gödel machine]] rigorously defines a specification for such an AGI.
 +
 
 +
== Development ==
 +
 
 +
Currently, there are no known Seed AIs in existence, but it is an active field of research. Several organizations continue to pursue this goal, such as the [http://intelligence.org Singularity Institute], [http://opencog.org/ OpenCog], and [http://adaptiveai.com/ Adaptive AI].
 +
 
 +
== See Also ==
 +
*[[Gödel machine]]
 +
*[[AI takeoff]]
 +
*[[Recursive self-improvement]]
 +
*[[Intelligence explosion]]
 +
*[[AGI]]
 +
 
 +
== References ==
 +
{{Reflist}}
 +
<ol start="4">
 +
<li>Yudkowsky, Eliezer.[http://intelligence.org/upload/LOGI/seedAI.html Seed AI Levels of Organization in General Intelligence]. Singularity Institute.</li>
 +
<li>[http://intelligence.org/files/GISAI.html#para_seedAI_advantage General Intelligence and Seed AI]. Singularity Institute.</li>
 +
<li>Schmidhuber, Jürgen. [ftp://ftp.idsia.ch/pub/juergen/gm6.pdf Gödel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements]. IDSIA</li>
 +
</ol>
 +
[[Category:AI]]
 +
[[Category:AGI]]

Latest revision as of 01:42, 22 June 2017

Smallwikipedialogo.png
Wikipedia has an article about


A Seed AI (a term coined by Eliezer Yudkowsky) is an Artificial General Intelligence (AGI) which improves itself by recursively rewriting its own source code without human intervention. Initially this program would likely have a minimal intelligence, but over the course of many iterations it would evolve to human-equivalent or even trans-human reasoning. The key for successful AI takeoff would lie in creating adequate starting conditions.

History

The notion of machine learning without human intervention has been around nearly as long as the computers themselves. In 1959, Arthur Samuel stated that "Machine learning is the field of study that gives computers the ability to learn without being explicitly programmed."[1] Since that time, computers have been able to learn by a variety of methods, including neural networks and Bayesian inference.

While these approaches have enabled machines to become better at various tasks[2] [3], it has not enabled them to overcome the limitations of these techniques, nor has it given them the ability to understand their own programming and make improvements. Hence, they are not able to adapt to new situations without human assistance.

Properties

A Seed AI has abilities that previous approaches lack:

  • Understanding its own source code. It must understand the purpose, syntax and architecture of its own programming. This type of self-reflection enables the AGI to comprehend its utility and thus preserve it.
  • Rewriting its own source code. The AGI must be able to overhaul the very code it uses to fulfill its utility. A critical consideration is that it must remain stable under modifications, preserving its original goals.

This combination of abilities would, in theory, allow an AGI to recursively improve itself by becoming smarter within its original purpose. A Gödel machine rigorously defines a specification for such an AGI.

Development

Currently, there are no known Seed AIs in existence, but it is an active field of research. Several organizations continue to pursue this goal, such as the Singularity Institute, OpenCog, and Adaptive AI.

See Also

References

  1. Yudkowsky, Eliezer.Seed AI Levels of Organization in General Intelligence. Singularity Institute.
  2. General Intelligence and Seed AI. Singularity Institute.
  3. Schmidhuber, Jürgen. Gödel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements. IDSIA