Difference between revisions of "Friendly artificial intelligence"

From Lesswrongwiki
Jump to: navigation, search
m (Undo revision 6028 by Russel08 (Talk) — revert spam)
m
Line 1: Line 1:
 
{{wikilink}}
 
{{wikilink}}
 
{{afwikilink}}
 
{{afwikilink}}
A '''Friendly Artificial Intelligence''' ('''FAI''') is an [[artificial general intelligence]] that has a positive rather than negative effect on humanity. ''Friendly AI'' also refers to the field of knowledge required to build such an AI. Note that ''Friendly'' (capital-''F'') is being used as a term of art, referring specifically to AIs that protect humans and [[humane]] values; an FAI need not be "friendly" in the conventional sense and need not even be sentient. Any AGI that is not friendly is said to be [[UnFriendly artificial intelligence|UnFriendly]].
+
A '''Friendly Artificial Intelligence''' ('''FAI''') is an [[artificial general intelligence]] that has a positive rather than negative effect on humanity. ''Friendly AI'' also refers to the field of knowledge required to build such an AI. Note that ''Friendly'' (capital-''F'') is being used as a term of art, referring specifically to AIs that protect humans and [[humane]] values; an FAI need not be "friendly" in the conventional sense and need not even be sentient. Any AGI that is not friendly is said to be [[Unfriendly artificial intelligence|Unfriendly]].
  
 
AI that underwent an [[intelligence explosion]] could exert unprecedented [[optimization]] power over its future; therefore, a Friendly AI could very well create an unimaginably [[Fun theory|good future]]. Conversely, an Unfriendly AI could represent an [[existential risk]]: destroying all humans, not out of hostility, but as a side effect of trying to do something [[Paperclip maximizer|entirely different]]. Just because an AI has the means to do something, [[Giant cheesecake fallacy|doesn't mean it will]].
 
AI that underwent an [[intelligence explosion]] could exert unprecedented [[optimization]] power over its future; therefore, a Friendly AI could very well create an unimaginably [[Fun theory|good future]]. Conversely, an Unfriendly AI could represent an [[existential risk]]: destroying all humans, not out of hostility, but as a side effect of trying to do something [[Paperclip maximizer|entirely different]]. Just because an AI has the means to do something, [[Giant cheesecake fallacy|doesn't mean it will]].
  
Requiring Friendliness doesn't make AGI any ''easier'', and almost certainly makes it harder. Most approaches to AGI aren't amenable to implementing precise goals, and so don't even constitute subprojects for FAI, leading to unFriendly AI as the only possible "successful" outcome. Specifying Friendliness also presents unique technical challenges: humane moral value [[Complexity of value|is very complex]]; a lot of [[Magical categories|seemingly simple-sounding moral concepts conceal hidden complexity]] not "[[mind projection fallacy|inherent]]" in the universe itself. It is likely impossible to specify humane values by explicitly programming them in, one needs a technique for extracting them automatically.
+
Requiring Friendliness doesn't make AGI any ''easier'', and almost certainly makes it harder. Most approaches to AGI aren't amenable to implementing precise goals, and so don't even constitute subprojects for FAI, leading to Unfriendly AI as the only possible "successful" outcome. Specifying Friendliness also presents unique technical challenges: humane moral value [[Complexity of value|is very complex]]; a lot of [[Magical categories|seemingly simple-sounding moral concepts conceal hidden complexity]] not "[[mind projection fallacy|inherent]]" in the universe itself. It is likely impossible to specify humane values by explicitly programming them in, one needs a technique for extracting them automatically.
  
 
==References==
 
==References==

Revision as of 08:10, 31 December 2011

Smallwikipedialogo.png
Wikipedia has an article about
Smallafwikilogo.png
The Transhumanist Wiki has an article about

A Friendly Artificial Intelligence (FAI) is an artificial general intelligence that has a positive rather than negative effect on humanity. Friendly AI also refers to the field of knowledge required to build such an AI. Note that Friendly (capital-F) is being used as a term of art, referring specifically to AIs that protect humans and humane values; an FAI need not be "friendly" in the conventional sense and need not even be sentient. Any AGI that is not friendly is said to be Unfriendly.

AI that underwent an intelligence explosion could exert unprecedented optimization power over its future; therefore, a Friendly AI could very well create an unimaginably good future. Conversely, an Unfriendly AI could represent an existential risk: destroying all humans, not out of hostility, but as a side effect of trying to do something entirely different. Just because an AI has the means to do something, doesn't mean it will.

Requiring Friendliness doesn't make AGI any easier, and almost certainly makes it harder. Most approaches to AGI aren't amenable to implementing precise goals, and so don't even constitute subprojects for FAI, leading to Unfriendly AI as the only possible "successful" outcome. Specifying Friendliness also presents unique technical challenges: humane moral value is very complex; a lot of seemingly simple-sounding moral concepts conceal hidden complexity not "inherent" in the universe itself. It is likely impossible to specify humane values by explicitly programming them in, one needs a technique for extracting them automatically.

References

Blog posts

See also