Difference between revisions of "Friendly artificial intelligence"

From Lesswrongwiki
Jump to: navigation, search
m (that you can't get there by luck is a conclusion down the road, not definition.)
Line 17: Line 17:
 
|url=http://yudkowsky.net/singularity/ai-risk}} ([http://intelligence.org/AIRisk.pdf PDF])
 
|url=http://yudkowsky.net/singularity/ai-risk}} ([http://intelligence.org/AIRisk.pdf PDF])
 
*[http://www.preventingskynet.com/why-we-need-friendly-ai/ Why We Need Friendly AI]
 
*[http://www.preventingskynet.com/why-we-need-friendly-ai/ Why We Need Friendly AI]
 +
 +
==Blog posts==
 +
*{{lesswrongtag|fai}}
  
 
==See also==
 
==See also==
Line 23: Line 26:
 
*[[Fun theory]]
 
*[[Fun theory]]
 
*[[Unfriendly artificial intelligence]], [[Paperclip maximizer]]
 
*[[Unfriendly artificial intelligence]], [[Paperclip maximizer]]
 
==Blog posts==
 
*{{lesswrongtag|fai}}
 
  
 
[[Category:Concepts]]
 
[[Category:Concepts]]
 
[[Category:Future]]
 
[[Category:Future]]
 
[[Category:AI]]
 
[[Category:AI]]

Revision as of 08:34, 5 January 2010

Smallwikipedialogo.png
Wikipedia has an article about
Smallafwikilogo.png
The Transhumanist Wiki has an article about

A Friendly Artificial Intelligence (FAI) is an artificial general intelligence that has a positive rather than negative effect on humanity. Friendly AI also refers to the field of knowledge required to build such an AI. Note that Friendly (capital-F) is being used as a term of art, referring specifically to AIs that protect humans and humane values; an FAI need not be "friendly" in the conventional sense and need not even be sentient. Any AGI that is not friendly is said to be UnFriendly.

AI that underwent an intelligence explosion could exert unprecedented optimization power over its future; therefore, a Friendly AI could very well create an unimaginably good future. Conversely, an Unfriendly AI could represent an existential risk: destroying all humans, not out of hostility, but as a side effect of trying to do something entirely different. Just because an AI has the means to do something, doesn't mean it will.

Requiring Friendliness doesn't make AGI any easier, and almost certainly makes it harder. Most approaches to AGI aren't amenable to implementing precise goals, and so don't even constitute subprojects for FAI, leading to unFriendly AI as the only possible "successful" outcome. Specifying Friendliness also presents unique technical challenges: humane moral value is very complex; a lot of seemingly simple-sounding moral concepts conceal hidden complexity not "inherent" in the universe itself. It is likely impossible to specify humane values by explicitly programming them in, one needs a technique for extracting them automatically.

References

Blog posts

See also